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Abstract 


This  research  accomplished  three  major  tasks.  First  it  examined  familiarity 
and  usage  rates  for  fifteen  published  R&D  project  selection  methods  in  the  context 
of  a  larger  general  issue,  the  Air  Force’s  ability  to  develop  and  exploit  technology. 
Wright  Laboratory  served  as  the  focus  for  the  research  effort  and  displayed  a 
greater  tendency  to  use  formal  methods  in  1993  than  was  shown  in  prior  research. 
The  study  identified  an  adaptive  organization  exercising  different  techniques 
through  the  years,  across  structure,  and  down  hierarchical  levels  to  meet  unique 
decision  environments.  An  overall  preference  for  simpler  models  like  Checklist, 
Scoring,  and  Sorting  models  led  to  a  recommendation  that  authors  familiar  with 
the  other  techniques  commtinicate  them  in  engineering  and  management  vernac¬ 
ular. 

Secondly,  the  study  introduced  a  technological  paradigm,  lateral  airfoils.  A 
bibliometric  search  for  patent  designs  dating  to  1910  suggested  a  sustained  trend 
in  the  technology’s  art  and  application.  The  methodology  employed  one  of  three 
new  lateral  airfoil  applications  introduced  in  the  narrative  to  meet  the  third  initia¬ 
tive. 

Finally,  the  study  uses  a  "placebo"  lateral  airfoil  research  project  to  gauge 
Wright  Laboratory’s  decision  making  process.  The  study  revealed  thirty  discrete 
criteria  and  successfully  reduced  these  to  seven  determinant  attributes  indicative 
of  overall  laboratory  support  for  applied  science  research  efforts. 
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RESEARCH  AND  DEVELOPMENT  PROJECT  SELECTION  TOOLS: 

PROBING  WRIGHT  LABORATORY’S  PROJECT  SELECTION 
METHODS  AND  DECISION  CRITERIA  USING  THE 
LATERAL  AIRFOIL  CONCEPT 

I.  IntrodnctiQii 

1.1  Introduction  to  the  Chapter 

This  chapter  introduces  the  investigation  topic  by  describing  the  general 
and  specific  issues,  research  objectives,  research  questions,  scope,  limitations  and 
assumptions.  A  summary  of  the  information  appears  at  the  end  of  the  chapter. 

1.2  General  Issue 

The  government  of  the  United  States  of  America’s  government  fimds  many 
types  of  research.  The  nation’s  investments  in  knowledge,  people,  educational 
pipelines,  facilities,  and  equipment  provide  extraordinaiy  benefits.  American 
science  and  technology  are  a  testament  to  the  nation’s  greatness. 

Our  national  piursuit  of  scientific  research  advances  at  a  phenomenal  rate 
on  two  fironts  simultaneously.  These  fironts  are  commonly  called  basic  and  applied 
research.  The  aim  of  basic  research  is  discovery  of  the  xmknown  that  is  accom¬ 
plished  by  scientists  who  explore  the  mysteries  of  their  discipline.  Applied  re¬ 
search  works  hand-in-hand  with  basic  research,  focusing  on  problems  in  using 
fundamental  concepts,  theories,  methods,  or  findings. 
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Throughout  the  1960’s  and  early  1970’s  the  United  States  Department  of 
Defense  (DoD)  supplied  the  lai^st  single  source  of  basic  and  applied  research 
funding  (OTA,  1991:8).  DoD  funding  levels  routinely  exceeded  National  Science 
Foundation  (NSF)  levels  by  a  factor  of  at  least  two  to  one.  Today,  Hod’s  spending 
level  is  £eu:  less  than  the  National  Institute  of  Health  (NIH),  who  serves  in  the  lead 
"funding  agency"  role.  Yet,  America’s  military  agencies  still  dispense  nearly  $3 
billion  annually  for  research  and  development  (R&D)  (OTA,  1991:102).  The  DoD 
distributes  funds  for  basic  and  applied  research  through  the  Army,  Navy,  Air 
Force,  Defense  Advanced  Research  Projects  Agency  of  1992  (DARPA)  or  Advanced 
Research  Prefects  Agency  of  1993  (ARPA),  and  Strategic  Defense  Initiative  Organi¬ 
zation  of  1992  (SDIO).  It  routinely  categorizes  R&D  funding  into  three  categories: 
6.1  -  fundamental  research  or  basic  research;  6.2  -  applied  research  and  exploratory 
development;  and  6.3A  -  the  initial  stages  of  advanced  development. 

DoD  and  the  national  scientific  community  as  a  whole  have  tremendous 
research  capability,  yet  there  are  more  basic  and  applied  research  opportunities 
than  the  country  can  resoiuroe.  The  Office  of  Technology  Assessment  (OTA) 
suggests  that  "The  objective,  then,  is  to  ensure  that  the  best  research  continues  to 
be  funded"  (OTA,  1991:7). 

07i4  finds  that  under  abnost  any  plausible  scenario  for  the  level  of  research 
funding  in  the  1990s,  there  are  issues  of  planning,  nutnagement,  and  progress  toward 
national  goals  to  address.  (OTA,  1991:10) 

The  difficult  task  becomes  reducing  the  list  of  potentially  fimitful  opportunities, 
finding  the  'best"  research  for  funding.  Aviation  Week  &  Space  Technology  hi¬ 
ghlighted  one  segment  of  government  facing  this  issue,  reporting,  "Proposals  for 
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the  Advanced  Research  Projects  Agency’s  Technology  Reinvestunent  Project  far 
outstrip  its  ability  to  fund  them”  (Lavitt,  1993:17). 

How  will  the  nation  effectively  allocate  its  R&D  resources?  How  can 
federal,  state  and  local  government,  academia,  and  industry  set  priorities?  These 
questions  are  as  hard  to  answer  as  they  are  easy  to  understand.  The  fluid  political 
environment  characterized  by  changing  funding  patterns  and  increasing  pressures, 
both  internal  and  external  to  the  national  research  system,  promises  a  complicated 
future  and  demands  a  coherent  policy. 

OTA  finds  that  Congress,  dte  executive  branch,  and  research  performers  must 
converge  on  these  issues.  Potential  congressional  actions  faB  inu>  three  categories. 

Congress  can:  I)  retain  primary  responsibility  for  decisions  and  initialing  actions;  2)  place 
some  of  die  responsibility  for  coonUnation  and  decisions  on  the  executive  branch;  and  3) 
encourage  research  performers  (especially  universities,  as  weU  as  Federal  and  industrial 
laboratories)  to  address  components  of  this  mue.  (OTA,  1991:43) 

This  thesis  meets  OTA’s  challenge  of  encoirraging  research  performers  to 
address  components  of  national  need  by  delivering  information  on  how  Wright 
Laboratory  selects  research  projects.  Further,  the  research  partially  alleviates  a 
separate  situation  identified  by  OTA  where  ". . .  most  analysis  and  research 
decision  making  must  draw  conclusions  firom  the  NSF  and  NIH  data  system” 

(OTA,  1991233). 

The  discussion  that  follows  addresses  the  general  issue  briefly  introduced  in 
the  preceding  paragraphs.  It  uses  four  fiictors:  Air  Force  Doctrine,  budget, 
mismanagement,  and  indigent  factors. 

l^.a  Air  Force  Doctrine.  The  forward  of  Air  Force  Manual  1-1, 

Basic  Aerospace  Doctrine  of  the  United  States  Air  Force,  asserts  that  the  text  is 
"one  of  the  most  important  documents  ever  published  by  the  United  States  Air 
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Force"  (AFMl-1, 1992:v).  This  statement  by  the  Chief  of  Staff  underscores  the 
manuscript’s  importance  and  signals  its  potential  for  framing  present  and  future 
Air  Force  policy.  Chapter  four  of  AFM  1-1  advocates  that  the  "Air  Force  should  be 
in  the^ibrefront  of  developing  and  exploiting  aerospace  power"  (AFMl-1, 1992:17). 
This  position  springs  from  a  wealth  of  historical  precedents  that  chronicles  re¬ 
search  and  development  efforts,  and  the  effects  of  technol<^0^s  exploitation. 
Traditionally,  these  elements  have  been  recognized  as  the  driving  force  that  pro¬ 
pelled  the  Air  Force  to  the  forefront  of  ^obal  aerospace  power.  Various  textbooks 
devoted  to  subjects  unrelated  to  war  or  aerospace  doctrine  also  acknowledges 
teclmol(^3r’s  significance.  Consider  the  following  quotation  from  an  Economics  text: 

Whether  a  firm  produces  textiles  or  toeomoHves,  whether  a  firm  is  big  or  small,  whether  a 
firm  is  run  by  a  genius  or  a  moron  (or  even  your  brotkerdn-law),  the  firm  cannot  do  more 
than  is  permitted  by  existing  technology.  (HiamsfiM,  1991:142) 

Technology  is  a  limiting  factor  in  commercial  business  and  war,  notwithstanding 

the  argument  that  war  is  a  business.  Yet,  recent  national  trends  threaten  the  Air 

Force’s  ability  to  push  technology’s  leading  edge. 

Sometiuus  du  advantage  of  a  superior  weapon  is  decisive  befitre  countermeasures  can  be 
evolved.  It  follows  then  dud  the  authods  used  to  select  and  develop  new  weapons  and  the 
doctrines  concerning  their  use  will  have  on  important  bearing  upon  the  success  or  failure  of 
Armies  -  oiuf  tf  nations.  (Holley,  1953:5-6) 

This  thesis’  general  issue  centers  on  the  Air  Force’s  abUity  to  develop  and  exploit 
technology.  R&D  project  selection  methodologies  and  criteria  employed  for 
resource  allocation  receive  particular  emphasis. 
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l^.b  The  BwHgttt  Faotnr.  The  current  United  States  economic 

condition  moderated  by  the  perceived  post-Cold  War  stability  is  forcing  law  makers 
to  make  tough  monetary  decisions.  Intense  budget  discussions  prompted  by  fiscal 
expenditure  constraints  initiated  in  the  early  1990’s  may  yield  lasting  effects. 
However, 

The  combination  of  tiu  most  obdurate  economic  recession  since  Ike  1930s,  the  severe  limits 
the  White  House  and  Congress  imposed  on  discretionary  speniBng  and  Ike  inability  to 
staunek  Ae  flow  of  red  ink  now  surest  that  a  budget  debacle  is  immineuL 

(Goodwin,  1992:55) 

Throughout  1992  and  early  1993,  many  senior  military  leaders  and  their 

civilian  coimterparts  suggested  a  high  potential  for  military  monetary  reductions. 

Many  lawmakers  pledged  their  support  of  efforts  to  pare  down  the  Pentagon 

budget  to  get  a  Teace  Dividend’  (Goodwin,  1992:57). 

With  Ike  end  of  the  Cold  War,  many  people  look  to  the  defense  budget  for  a  peace  dividend 
-that  is,  for  funds  dial  can  be  redirected  to  other  purposes  including  domestic  programs,  tax 
cuts,  and  deficit  reduction.  (CBO,  1992:61) 

Major  General  Smith,  Chief  of  Staff  for  Logistics,  United  States  Air  Force  Material 
Command  suggested  in  1992  that  the  Air  Force  could  expect  reductions  greater 
than  or,  at  best,  equal  to  25  percent  in  this  environment  (Smith,  1992).  Large 
segments  of  the  Air  Force,  including  its  Research  and  Development  (R&D)  organi* 
zationj  and  extended  civilian  contracting  community,  will  experience  the  repercus¬ 
sions  firom  down-sizing  actions  of  this  magnitude.  Initially,  the  R&D  community’s 
reaction  included  ’ztA>-sum’  budgeting,  where  new  prefects  are  offoet  hy  cuts  in 
older  projects  while  maintaining  a  certain  level  of  fiscal  expenditure.  Dr  Allan 
Bromley,  the  President’s  science  advisor,  extolled  this  position  during  the  twilight 
of  President  Bush’s  administration  (Cioodwi'^ ,  1992:57). 
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As  the  competitiaa  for  money  escalates  throughout  the  1990’s,  creative 
reactions  may  emerge.  Intense  competition  for  scarce  budgetary  funding  surfaced 
in  the  R&D  community  as  early  as  December  7, 1992  when  Aviation  Week  & 
Space  Technology  published  its  cover  story  titled,  "U.S.  Labs  Reorient  to  New 
Endeavors.”  The  following  article  excerpt  records  the  laboratories’  public  an¬ 
nouncement  of  refocusing  efforts  in  the  wake  of  a  new  world  order. 

The  US  Govenuumt's  network  of  defense-retated  laboratories  is  atyusting  to  the 
post-CoU  Vfar  world  in  muek  the  same  way  as  the  aerospace  industry.  The  labs  are 
focused  on  an  oggresshe  search  for  new  financial  alliances . . .  (Pulghum,  1992:46) 


A  follow-up  article  published  as  part  of  a  Global  Aerospace/Defense  Industry 
5  Year  Outlook  reafiGrmed  initial  refocusing  activities. 

Defense  RAD  will  not  be  short-changed  over  the  next  five  years,  but  will  undergo 
drastic  restructuring.  Energy  Dept  laboratories  will  be  chaUet^ed  to  find  industrial  partners 
to  share  development  costs  and  revenues  or  go  out  of  business.  However,  as  they  seek  closer 
ties  with  industry,  the  labs’  risk  losing  some  of  their  best  scientists  and  engineers  to  higher 
paying  commercial  partners.  Competidon  among  laboratories  will  be  a  way  ofUfe  as  they 
scramble  for  both  military  and  commercial  funds.  (Scott,  1993:59) 


Inevitability,  military  R&D  prqjects  and  portfolio’s  may  face  reductions  in  scope, 
extensions  in  project  lead  times,  and  possible  elimination.  Unfortunately,  these 
actions  will  negatively  affect  many  potentially  ffuitfol  research  technologies.  A 
Business  Week  journalist  writes: 

For  the  first  time  in  years,  a  brood  array  of  science  projects  -from  the  space  station 
to  efforts  to  boost  commercial  technology  could  be  vulnerable  to  bi^et-cutters  on  Capitol 
HttL  (Carey,  1992:66) 


Even  relatively  small  Air  Force  projects  will  undoubtably  come  under  the 
same  scrutiny.  This,  then,  poses  a  dilemma  for  senior  decision  makers  who 
function  under  Air  Force  guidance.  On  which  visiou  do  they  focus?  Should  a 
decision  maker  place  equal  importance  on  more  than  one  goal?  How  does  a 
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dedsioa  maker  ensure  the  Air  Force  remains  in  the  forefiont  of  aerospace  technolo¬ 
gy  while  there  are  other  competing  goals,  like  staying  within  a  budget? 

Althouid^,  the  Air  Forces'  basic  aerospace  doctrine  reo^piizes  a  fluid 
environment,  R&D  monetary  reductions  ultimately  make  the  d3nuunic  tedmoli^ 
exploration  pass6  while  exacerbating  tradeofis,  "such  as  those  between  mass  and 
flexibility,  functional  excellence  and  versatility,  and  predictability  and  adaptability" 
(AFM  1-1, 1992:225). 


1J2.C  The  Mismanagement  Factor.  Increasingly,  media's 

attention  has  focused  on  R&D  mismanagement.  The  broad  spectrum  of  personality 
types  and  large  sums  of  money,  $43.3  billion  in  R&D  for  fiscal  year  1993  (FY93) 
(Goodwin,  1992:57),  makes  R&D  a  "target  rich"  environment  for  mismanagement. 
In  its  most  disgracefully  corrupt  form,  the  mismanagement  cry  accompanies  public 
scandal. 

. . .  scondob  over  Ou  misase  of  federal  research  dollars  at  Universities  such  os 
Sittttford  and  aUegadoas  efmuseonduct  in  prominent  labs  have  hurt  science’s  iuu^e. 

(Carey,  1992:68) 


Do  senior  corporate  executives  perceive  mismanagement  in  peacetime  in  the  same 
light  as  mismanagement  on  the  battlefield?  Hughes  Aircraft  Company’s  Chairman 
and  Chief  Executive  Officer  notes  that. 

Peace  in  our  time  creates  a  new  world  order  with  complexities  and  dangers  of  its 
own.  Unless  we  understand  the  peace  and  learn  to  survive  in  it,  we  as  a  nation  can  lose 
even  more  in  terms  of  our  vitality  and  standards  ofUving  as  from  defeat  in  an  actual 
shooting  war.  (Currie,  1991:50) 
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Whispers  of  mismanagement  often  bubble  to  the  surface  in  subtle  forms, 
shrouded  in  funding  debates.  Consider  the  following  excerpt  that  discusses  a 
seemingly  ’qualify  based’  decision. 

. . .  drug  eompoMks  tune  betn  scaHmg  back  reuareh  on  mUiMoties  and  edur  antumerobuds 
and  tanimg  instead  to  andcancer  and  andviral  drugs.  So  whan  drug-resistant  TB  first  came 
to  nadanai  attendon  in  1991,  dure  was  no  VS.  source  for  two  TB  drugs  that  were  fornurfy 
used  to  treat  du  tBsease  . . .  Thomas  L.  Copaeann,  head  of  the  bbhtechnology  of  the 
Pharmaeeudeal  Mangfaeturers  Assodadon,  places  du  Name  on  the  ’dismal’  amount  being 
spent  on  drug  resistance  by  the  Nadonai  Insdtute  of  Health.  (Beardsley,  1992:20) 

While  the  writer  may  be  advocating  the  need  for  hi^er  funding  levels,  one  could 
also  imply  a  criticism  for  mismanagement  of  current  funding  levels.  Strong, 
quality  management  tools  and  methodologies  serve  as  the  best  protection  against 
the  mismanagement  stigma.  Primarily,  an  appropriate  array  of  these  elements 
enables  superior  R&D  portfolio  decisions  and  minimixe  opportunities  for  miticasm. 
However,  if  questions  do  arise,  they  function  as  the  bulwark  in  the  debate  over 
how  to  "get  the  most  bang  for  the  buck!" 

l^.d  The  USAF  Indigent  Factor.  R&D  decision  makers  need 

an  assortment  of  tools  to  discriminate  between  research  projects.  The  typical 
USAF  R&D  Manager  retains  responsibility  for  evaluating  project  research  poten¬ 
tial,  comparing  prqject  feasibility,  selecting  appropriate  research  technologies,  and 
securing  or  distributing  research  funding.  Past  Air  Force  sponsored  research 
revealed  that  decision  makers  relied  on  "intuition  and  expert  judgment"  as  the 
primary  means  for  project  selection  (Congdon,  1988:69).  Other  studies  have 
confirmed  this  observation  as  DOD  standard  practice  (Robinson,  1991:576). 
Additionally,  the  USAF  has  followed  the  governmental  scientific  research  commu- 
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nity  as  a  whole  by  not  adopting  or  publishing  decision  making  criteria  for  research 
project  discrimination. 

AUtough  both  D.  Alim  BnmUy,  FrtsUetit  Buskos  science  adviser,  and  Frank 
Press,  gresUent  of  the  National  Academy  of  Scknees,  have  idenl^d  criteria  and  types  of 
priorities  that  they  consider  essentiaifor  scknee,  neither  addresses  the  probkm  that  there  is 
few  mechanisms  for,  and  no  tradition  of,  ranking  research  topks  across  vaious  fields  and 
sutfklds  of  laf airy.  In  odeBdon,  priority  setting  is  often  resiskd  by  recounts  of  federal 
fiends  who  fear  the  loss  of  support  for  their  partkular  speciaBty. 

(Chubin  A  Robinson,  1991:B2) 

However,  a  caution  is  in  order.  This  type  of  public  criticism  does  not  prove  gross 
error  or  negligence.  In  fact,  it  highlights  a  range  of  problems  encountered  by 
ranking  research  topics  identically  across  subfields  of  inquiry.  The  following  para¬ 
graphs  animate  three  views  in  the  debate. 

First,  consider  the  case  of  a  large  research  facility  like  Wright  laboratory. 
The  oi^[anizational  structure  for  overseeing  three  thousand  people  requires  a 
hierarchy  with  laboratory,  directorate,  branch,  and  section  tiers.  Should  decision 
makers  use  the  same  methods  or  criteria  at  all  organizational  levels?  Do  decision 
makers  need  to  compare  research  projects  at  the  laboratory  or  aggregate  level 
using  the  same  determinant  attributes  that  have  been  used  for  ratings  at  discrete, 
lower  levels?  These  questions  echo  of  the  proverbial  comparison  of  apples  to 
oranges.  A  comparison  usually  avoided  at  all  costs,  but,  which  may  be  okay  in 
special  cases. 

For  example,  assume  a  grocery  store  manager  and  homemaker  are  interest¬ 
ed  in  quality.  When  the  manager  performs  a  quality  inspection,  it  makes  little 
difference  if  the  object  is  an  apple  or  an  orange.  The  manager's  only  interest  is  in 
the  quality  of  the  fruit.  However,  a  homemaker  who  is  interested  in  apples  and 
oranges,  proceeds  under  a  different  agenda,  (i.e.  is  the  apple  a  high  quality 
Winesap  or  a  poor  quality  Johnathan  apple?).  In  other  words,  the  store  manager’s 


9 


determinant  attributes  for  quality  may  have  little  to  do  with  the  homemaker's 
determinant  attributes  for  quality.  Bringii^  this  example  back  to  the  R&D  debate, 
one  can  see  that  the  laboratory’s  criteria  for  a  portfolio  candidate  may  differ 
substantially  from  any  discrete  branch’s  criteria  for  an  R&D  proposal. 

Secondly,  identical  decision  making  criteria  may  not  universally  apply 
between  laboratories  and  directorates.  For  example,  Wright  Laboratory’s  Materials 
Directorate  may  use  its  own  distinctive  set  of  determinant  attributes  while  the 
Propulsion  Directorate  may  tise  another  set.  It’s  not  that  one  directorate  labels  its 
sister  directorates’  criteria  as  imimportant;  it’s  simply  that  their  criteria  may  not 
apply.  Trying  to  force  a  particular  criteria’s  usage  across  a  laboratory’s  director¬ 
ates  may  present  a  oontinuvim  of  problems  ranging  firom  inconvenience  at  the 
branch  or  division  level  to  impracticability  at  the  laboratory  decision  making  level. 
Therefore,  a  laboratory  management’s  enforcement  efforts  for  exactly  the  same 
decision  making  criteria  may  simply  be  fruitless. 

Thirdly,  a  £edr  introduction  of  the  topic  as  it  relates  to  the  Air  Force  arena 
requires  a  discussion  of  past  practices.  Prior  Air  Force  graduate  research  studies 
on  this  issue  document  that  both  prqjects  and  portfolios  are  compared.  However, 
the  lack  of  any  comparison  method  would  come  as  no  surprise,  since  surveys  at 
both  military  and  civilian  institutions  docxxment  R&D  decision  makers’  reluctance 
to  adopt  systematic  project  assessment  methodologies  (liberatore,  Titus,  1983:962; 
Prince,  1985:40;  Congdon,  1988:73-4).  The  research  of  Prince  (Prince,  1985)  and 
Congdon  (Congdon,  1988)  firmly  communicate  that  R&D  comparisons  are  indeed 
made  at  many  bureaucratic  levels  within  Wright  Laboratory  at  Da3rton,  OH. 
However,  their  research  also  suggests  that  the  methodologies  and  determinant 
attributes  may  vary  between  directorates. 
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After  reviewing  the  three  tenets  presented  above,  it  should  be  apparent  that 
there  are  many  reasons  why  differences  in  methodologies  exist  and  why  standards 
for  research  prqject  comparison  at  any  discrete  level  throu^out  the  laboratory  do 
not  exist. 


lJ2,e  General  Issn^  Snnmiary.  Air  Force  Doctrine,  the  national 

budget,  the  potentials  for  mismanagement,  and  indigent  factors  collectively  suggest 
a  real  and  current  need  for  research  and  development  project  selection  methodolo¬ 
gies  in  DoD.  Wright  Laboratory  has  been  chosen  as  the  research  institution  for 
this  study.  Publication  of  the  discrete  criteria  and  aggregate  comparison  method¬ 
ologies  currently  used  may  serve  as  a  model  for  other  military,  civilian,  institu¬ 
tions.  Section  1.3  briefly  introduces  an  additional  study,  lateral  airfoils  that 
exemplify  a  technological  paradigm  and  is  referenced  extensively  throughout  the 
thesis. 


1.3  Lateral  Airfoils 

'Hiis  thesis  uses  lateral  airfoils  to  secure  infonnation  about  Wright  Labora¬ 
tory  R&D  portfolio  selection  methodology.  Lateral  airfoils  serve  as  a  "research 
project  placebo"  during  the  interview  process  described  in  Chapter  III.  An  under¬ 
standing  of  lateral  airfc^  technology  is  needed  before  discussing  the  specific  issue, 
research  objectives,  and  research  questions. 

Lateral  airfoils  typify  a  technological  paradigm,  a  new  technology  that  the 
Air  Force  coiild  research,  but  which  is  not  currently  being  researched.  Basically,  as 
an  aerospace  vehicle  design,  they’re  easily  understood  through  a  comparison  with 
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helicopter  main  rotor  blades.  Helicopter  blades  "rotate"  in  a  plane  above  the 
fuselage,  or  body  of  the  hehcopter.  They  provide  lift  and  thrust  to  sustain  flight. 
Lateral  airfoils  perform  the  same  function,  but  in  a  different  fashion.  They 
"revolve"  in  a  cylinder  around  or  outboard  of  the  fuselage. 

This  thesis  marks  the  first  formal  introduction  of  lateral  airfoils  to  the 
USAF  research  community.  Chapter  II  summatizes  three  applications  for  lateral 
airfoil  technolc^  while  Appendix  A  provides  details  of  several  patents.  Technical 
applications  of  this  technology  include,  but  are  not  limited  to,  the  following 
examples: 

1.  A  new  aerospace  vehicle  encompassing  the  qualities  of  a  helicopter 
without  the  limiting  effects  on  forward  flight  velodly  resulting  from  retreat¬ 
ing  blade  stall. 

2.  A  new  gas  turbine  engine  design  facilitating  combined  cycle  engine 
technologies  through  combinations  of  lateral  airfoil  compressor  and  turbine 
sections  with  RAM  jet  engine  technology. 

3.  A  new  naval  propulsion  system  with  applications  on  submeurine,  siirface 
ship,  and  hydrofoil  craft. 

To  date,  there  have  been  many  United  States  government  patents  granted 
for  lateral  airfoils,  albeit  \mder  many  names.  Lateral  airfoil  technology,  if  consid¬ 
ered  as  a  serious  research  candidate,  would  compete  with  both  present  and  future 
Wright  Laboratory  projects.  Given  limited  research  resources  like  facilities,  work 
force,  and  monetary  funding,  senior  Wright  Laboratory  managers  would  either 
discard  lateral  airfoil  technology,  table  research  until  some  later  date,  or  include  it 
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in  a  research  program  portfolio  of  some  type.  This  research  explores  how  Wright 
Laboratory  approaches  a  decision  on  technologies  like  lateral  airfoils. 

1.4  Definition  of  Terms 

An  important  part  of  understanding  the  specific  issue  and  research  ques¬ 
tions  is  a  grasp  of  the  terminology  commonly  used  by  practitioners  in  the  field  of 
complex  decision  making.  The  following  list  includes  words  and  phrases  fiam 
Analyzing  Decision  Making:  Metric  Conjoint  Analysis  (Louviere,  1988:12)  which  are 
used  in  the  remainder  of  this  thesis. 

Physical  Variable:  Observations  or  measurements  of  various  physical 

antecedents  of  determinant  attributes. 

Determinant  decision  criteria  used  by  decision  makers 
to  evaluate  research  projects. 

Beliefs  that  decision  makers  have  about  the  amount  of 
each  determinate  attribute  possessed  by  the  research 
project. 

Overall  Utilities:  The  judgments,  impressions,  or  evaluations  that 

decision  makers  form  on  research  projects,  taking  all 
the  determinate  attribute  information  into  account. 

Final  Choice  Set:  The  set  of  all  potential  research  project  discrimination 

criteria  seriously  considered  prior  to  a  decision  maker^s 
choice. 


Attributes: 


Position: 
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Choice: 


The  cognitive  process  by  which  a  decision  maker,  after 
evaluation  all  the  possible  viable  research  prqjects, 
decides  to  select  one  project,  a  portfolio,  or  to  make  no 
choice. 

Part-Worth  Utility:  Judgments  decision  makers  perform  about  "how  good,” 

"how  satisfactory,"  or  "how  whatever"  particular 
positions  of  particular  products  might  be  on  particular 
determinant  attributes. 


1.5  Specific  Issue 

In  a  generic  sense,  the  specific  issue  concerns  the  usefulness  of  cost  esti¬ 
mates  when  compared  with  all  possible  factors  considered  in  a  research  portfolio 
decision.  The  exacting  language  used  with  complex  decision  making,  detailed  in 
section  1.4,  firames  the  specific  issue  as  follows: 

Specific  Issue:  Considering  the  proliferation  of  determinate  attributes 
available  to  the  decision  makers,  what  is  the  part-worth  of  the  cost  estimate  in  a 
lateral  airfoil  research  initiation  final  choice  set? 


1.6  Research  Objectives 

This  thesis  investigates  the  specific  issue  within  the  bounds  of  three 
research  objectives.  Again,  the  strict  language  of  complex  decision  making  is  used. 
However,  the  objectives  simply  map  the  tools  decision  makers  use  to  the  degree 
used  and  addresses  the  usefulness  of  the  engineering  development  cost  estimate  (a 
specific  tool)  in  a  research  portfolio  decision. 
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Objective  1:  Solicit  the  determinate  attributes  that  USAF  decision  makers 
use  to  discriminate  between  research  projects  with  exceptional  potential. 

Objective  2:  Evaluate  the  determinant  attributes  espoused  by  the  dedsion 
makers  by  developing  and  describing  a  concept  that  Wright  Laboratory  as  a  whole 
has  not  previously  researched. 

The  iise  of  an  independent  mechanism,  "research  project  placebo,”  forms  the 
basis  for  the  introduction  and  discussion  of  lateral  airfoils. 

Objective  3:  The  final  objective  is  to  seek  a  measure  of  an  engineering 
development  cost  estimate’s  part-worth  utilily  in  Wri^t  Laboratory’s  set  evalua¬ 
tion  criteria.  In  effect,  to  find  the  usefulness  of  the  estimate.  This  objective  is 
aided  by  using  the  placebo  research  project,  lateral  airfoils,  to  minimize  the  effect 
of  familiarity  with  the  technology. 

1.7  Research  Questions 

The  research  questions  illuminate  the  research  objectives  by  focusing  the 
thesis  methodology  on  the  objectives  and  guiding  the  research.  The  first  question’s 
language  matches  question  one  firom  Congdon’s  thesis  (Congdon,  1988:7). 

Question  1:  What  methods  do  managers  use  to  select  R&D  projects  and 
allocate  laboratory  resoiirces  at  Wright  Laboratories? 

Question  2:  What  establishes  lateral  airfoils  as  a  potentially  viable 
research  project? 
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Question  3:  What  determinate  attributes  do  decision  makers  use  in  the 
final  choice  set  during  research  program  portfolio  selection? 

Question  4:  What  is  the  part-worth  utility  of  an  engineering  development 
cost  estimate  in  the  decision  maker’s  choice  on  a  lateral  airfoil  design? 

1.8  Scope 

Currently  the  Air  Force  conducts  research  at  four  laboratories  -  Rome 
Laboratory,  Phillips  Laboratory,  Armstrong  Laboratory,  and  Wri^t  Laboratory 


Figure  1.  Air  Force  RSO  Qrgaziizatioas  (Hrxght  Laboratory  Fact 
Sheet,  1992) 


(see  figure  1).  The  Air  Force  routinely  sponsors  research  at  institutions  outside  the 


military  environment  that  includes  other  government  agencies,  civilian  imiver- 
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sities,  and  public  and  private  organizations.  Within  the  Air  Force,  each  lah  has 
many  directorates  who  manage  both  in-house  research  and  research  contracts  with 
outside  agencies. 

This  research  investigates  the  decision  making  tools  used  in  the  R&D 
project  selection  process  by  decision  makers  within  Wright  Laboratory.  The 
laboratory  has  seven  directorates  like  the  Aero  Propulsion  and  Power  directorate, 
the  Flight  Dynamics  directorate,  or  the  Materials  directorate,  each  with  decision 
makers  at  several  levels:  the  Division  Chief,  Chief  Scientist,  Deputy  Director,  and 
Branch  level.  Since  the  population  of  decision  makers  outside  the  Wright  Labora¬ 
tory  complex  is  extensive,  this  research  tises  only  a  representative  cross-section  of 
Wright  Laboratory  decision  makers  at  Wright-Patterson  Air  Force  Base,  Ohio  and 
Eglin  Air  Force  Base,  Florida. 

Limitations 

This  research  is  limited  to  exploring  the  factors  used  to  discriminate  among 
potential  research  projects  for  Air  Force  categories  of  fimding  t3rpically  called 
applied  research:  Research,  Exploratoiy  Development,  and  Advanced  Development 
categories.  Since  research  project  proposals  are  initiated  from  a  variety  of  soiurces 
and  levels  within  the  government,  this  research  fiurther  limits  itself  to  the  criteria, 
tools,  format,  and  methodology  that  current  laboratory  managers  use  to  dispense 
resources  under  their  control. 
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1.10  AgMomptiowfi 

This  research  assumes  that  decision  makers  will  either  consciously  or 
subconsciously  display  bias  when  discussing  projects  under  their  management. 
Additionally,  some  Air  Force  projects  are  classified  or  subject  to  proprietary  sensi¬ 
tivity.  Lateral  airfoils  represent  a  unique,  unclassified  research  prqject  that 
relieves  the  tensions  created  by  these  asstunptions. 


1 .11  fiiiminarv 

Chapter  I  introduced  the  investigation  h  \  It  described  the  general  issue 
that  is  at  the  heart  of  the  Air  Force’s  ability  to  develop  and  exploit  technology,  llie 
discussions  included  the  effect  of  budget,  mismanagement,  and  Air  Force  need  on 
the  general  issue.  The  thesis’  specific  issue,  research  objectives  and  questions  were 
all  detailed  using  complex  decision  making  language.  The  assiunptions,  scope,  and 
limitations  of  the  research  completed  the  chapter. 

While  Chapter  1  laid  the  foundation  for  the  literattire  review.  Chapter  II 
examines  a  variety  of  literat\ire  sources  indudii^  Naval  and  Air  Force  research  on 
DOD  laboratory  R&D  project  selection  methodologies,  patents,  published  sources, 
and  artist  conceptions.  It  develops  and  explains  the  concept  basis  for  lateral  airfoil 
technology  and  also  includes  an  overview  of  R&D  project  selection  methodologies. 

Chapter  III  details  the  investigative  methodology  for  examining  the  specific 
issue  and  four  research  questions.  Each  of  the  research  questions  required  a 
tailored  methodology  that  is  addressed  separately  in  Chapter  in. 

Chapter  IV  contains  a  discussion  and  analysis  of  the  research  results.  The 
narrative  begins  with  a  consideration  of  the  laboratory’s  response  dem(^;raphics. 
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including  parallel  research  demographics  firom  1985  and  1988.  Hie  results 
embrace  familiarity  and  usage  patterns  within  Wri|^t  Laboratory  with  published 
R&D  prqject  selection  methodok^es.  Chapter  IV  ends  with  an  assessment  of  the 
part'worth  of  an  engineering  development  cost  estimate,  as  a  decision  criteria,  and 
a  discussion  of  the  research  merit  of  lateral  airfoil  technology. 

Chapter  V  completes  the  report  of  research  conducted.  It  contains  research 
conclusions  and  recommendations  for  future  studies. 
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n.  Literature  Review 

2A  Introduction 

This  literature  review  examines  four  discrete  topics.  Wright  Laboratory's 
organizational  struct\ire  and  mission  serve  as  the  first  topic  of  investigation.  Later 
discussions  of  laboratory  methodologies  and  criteria  build  upon  an  imderstanding 
of  laboratory  responsibilities  for  its  decision  making  environment 

The  second  topic  discussed  in  section  three  of  this  chapter  constructs  a 
firamework  for  examining  R&D  project  selection  techniques.  Discussion  of  the  tech¬ 
nique  selection  issue  and  an  overview  of  the  nature  and  form  of  common  method- 
olives  is  provided.  However,  the  section  references  Appendix  A  for  details  of  the 
different,  published  methods. 

A  Historical  Perspective,  section  2.4,  reassesses  prior  studies  on  Wright 
Laboratory  R&D  prqject  selection  techniques  and  factors.  The  section  incoiporates 
the  research  findings  of  Brooks  (1979),  who  modeled  the  laboratory  decision 
process.  The  section  also  considers  Congdon  (1985)  and  Prince’s  (1988)  investigat¬ 
ion  of  the  use  of  R&D  prcgect  selection  tools. 

A  fourth  topic  is  covered  in  section  2.5,  Lateral  Airfoil  Tpnhnology.  which 
examines  potential  uses  for  lateral  airfoils.  Since  many  technological  applications 
exist,  the  discussion  is  limited  to  one  design  each  for  an  aerospace  vehicle,  gas 
turbine  engine,  and  naval  propulsion  system.  The  section  includes  drawings  and 


artist  sketches. 


2J2  Wright  Laboratory 

Wright  Laboratory’s  heritage  for  advancing  aviation  technology  dates  firom 
the  very  dawn  of  powered  aviation.  It  begins  in  the  1900’s  when  the  Wright  flyer 
thrust  into  the  air  over  the  rolling  hills  of  America’s  heartland  near  Dayton,  Ohio. 
The  same  ground  that  saw  some  of  mankind’s  earliest,  struggling  airborne  efforts, 
today  holds  a  large  government  owned  and  operated  research  laboratory  complex 
dedicated  to  continuing  mankind’s  adventures  into  the  "wild  blue  yonder.” 

Wright  Laboratory  is  one  of  the  four  United  States  Air  Force  "Super 
Laboratories"  which  falls  under  the  purview  of  the  Aeronautical  Systems  Center 
(ASC).  ASCs  existence  began  on  July  1, 1992  after  the  Air  Force  combined  its 
System  Command  and  Logistics  Commands  into  a  single  entity.  The  merger  led  to 
reorganisation  and  subsequent  renaming  of  the  former  Aeronautical  Systems 
Division  to  the  current  Aeronautical  Systems  Center.  Today,  each  of  ASCs  four 
laboratories  focuses  on  tmique  research  disciplines.  Specifically,  the  Wright 
Laboratory  mission  is  to 

. .  Jead  atid  focus  the  Air  Force’s  aerouaudeal  techuology  investment  by  performing  inkouse 

research  and  establishing  contractual  parourships  with  universities  and  industry  to: 

thscover  enabUng  technologies  that  offer  potential  for  revoludonary  improvements 
in  the  performance,  affordabWty,  and  supportaNhty  of  Air  Force  weapon  systems. 

develop  and  demonstrate  advanced  technologies  for  both  current  and  future  Air 
Force  weapon  systems  to  best  meet  our  users’  needs. 

transition  proven  technologies  to  weapon  system  developers  and  maintainers  in  an 
aggressive,  expeditious  manner. 

solve  pressing  technical  problems  wherever  they  occur  through  respondve  support 
to  any  Air  Force  organisation,  24  hours  a  day,  in  time  of  peace  or  war. 

(Wright  Laboratory  Fact  Sheet,  1992) 
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Figure  2.  Wrlght  Labormtory  Ox<gaalz«ti.ott  (Fact  Shaaft,  1992) 

Today’s  orgazazatimal  structure  for  Wngiit  Laboratory  shows  three  support 
directorates,  seven  technology  directorates,  and  a  strategic  planning  directorate. 
Together,  the  command  section  and  directorates  employ  over  three  thousand  people 
(Wright  Laboratory  Fact  Sheet,  1992).  The  1993  organizational  chart  s^jpears  in 
Figure  2.  The  directorates  operate  in  a  united,  synergistic  effort  meeting  the 
overall  mission  through  application  of  their  discrete  specialties.  A  description  of 
each  directorate’s  taskings  follows: 


The  Materials  Directorate  explores  new  autcrials  and  processes  for  advanced 
aerospace  appBcations.  Its  cnrrent  focus  is  on  tfaemial  protection  materiais,  metallic  and 
nonmetalic  structural  matariab,  aerospace  propulsion  materials,  electromagnetic  and 
electranic  materials  and  laser-hardened  materials. 


The  SoBd  State  Electronics  Directorate  is  respon^le  for  electronic  device  research  and 
devdopmcnt  in  the  areas  of  microelectronics,  microwaves  and  electro-optics.  Research 
extends  from  ftmdamental  semiconductor  lajer  growth  and  device  fobrication  through 
integrated  carcohs.  In  the  electro-optks  area,  lasers,  detectors  and  int^rated  focal  plane 
arrays  are  developed. 


4 


The  MannCKtiiriim  Tedmology  Directorate  serves  as  the  focal  pout  for  ptaaaiBg  ami 
exccath^  an  integrated  mannfactiiring  program  across  the  Air  Force.  In  addition  to  a 
focm  on  nuonfheturing  larocess  technologies  and  coupidcr  intended  manoCactnring,  the 
directorate  also  focuses  on  design  for  producibility,  design  for  quality,  and  des^  for  life 
•  cyde  costs,  otherwise  known  as  int^rated  product  developnicnt 

The  Aero  Propulsion  and  Power  Directuate  focuses  on  airbreathing  propulsion  and 
aerospace  power  technology,  which  includes  fuels  and  lubricants,  turbine  engines,  and 
h4[h  performancc/high  inacta  air-breathing  propulsion  applications.  Aerospace  power 
research  up  to  megawatt-class  systems  centers  around  electro-chemkal  energy  storage, 
hyperconducting  generators,  and  power  conditioning  subsystems. 

The  Avionics  Directorate  conducts  research  and  development  activities  m  the  fields  of 
offensive  sensors  (e.g.,  radar,  infhured  search  and  track,  forward  looking  infknred), 
weapon  delivery  systems,  reconnaissance,  electrooic  warfare,  navigation,  communications, 
and  avionics  integration. 

The  Armamoit  INrectorate  develops  conventioiial  armament  technologies  and  integrates 
those  into  air-vehkle  and  other  delivery  platforms.  The  directorate  provides  conventional 
armament  technology  for  four  mqjor  t^sts  which  include  advanced  gaidanoe,  weapon 
flight  mechanics,  ordnance,  and  strmegk  defense. 

The  Fl%ht  Dynamics  Directorate  conducts  the  Dili  spectrum  of  flight  vehicle  research. 
Primary  areas  of  interest  include  structures,  vehicle  subsystems  (landing 
gear,  transparencies,  etc),  flight  control  and  aeromechanics.  In  additioo,  this  directorate 
develops  and  maintains  a  fleet  of  experimoital  test  vehkies  to  demonstrate  integrated 
technologies  -avionics,  propulsion  —  in  an  airborne  environment 

(Wright  Laboratory  Fact  Sheet  1992) 


Wright  laboratory  wielded  a  financial  budget  in  excess  of  $950  milbon  for 
fiscal  year  1992,  spending  over  $700  milbon  on  1333  active  contracts  with  300 
industrial  and  academia  associates.  Figure  3  shows  that  76  percent  of  this  money 
funded  contract  work  outside  the  laboratory.  Additionaby,  the  laboratory  evalu¬ 
ated  and  influenced  "over  3,500  contractor  Independent  Research  and  Development 
(IR&D)  projects  involving  approximately  $2  B  annuaUy"  (Wright  Laboratory  Fact 
Sheet,  1992). 

Wright  Laboratory  statistics  for  1992  include  over  3090  personnel  assigned 
with  a  ratio  of  an  estimated  84  percent  dviban  and  16  percent  military  research- 
force.  Less  than  one  hundred  enbsted  servicemen  formed  the  laboratory’s  ranks. 
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Not  surprisingly,  Figure 
4  shows  80  percent  of  the 
personnel  hold  an  engi¬ 
neering  degree  of  some 
speciality  with  an  addi¬ 
tional  15  percent  pos¬ 
sessing  math  and  science 
d^irees.  'Hiese  smns  in¬ 
clude  283  technicians 
and  nearly  two  thousand 
scientists  and  engineers. 


Figure  4  .  Wrlght  Laboratory  Denograpbi.cs  (Fact  Sheet,  1992) 
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2.3  R&D  Project  Selection  Techwinnea 


Why  use  an  R&D  project  selection  technique?  Are  there  diiOferent  decision 
techniques  available?  To  what  degree  is  Air  Force  management  interested  in  R&D 
project  selection  techniques?  Which  method  is  the  best?  These  are  all  questions 
relevant  to  the  thesis  specific  issue  and  the  R&D  community  as  a  whole.  This 
chapter  introduces  several  techniques  and  notes  their  usefulness  while  Appendix  A 
contains  an  informative  overview  of  each  technique. 

Appendix  A  describes  a  variety  of  techniques  using  management  style 
terminologies  without  using  complicated  mathematical  formulas.  It  skims  the 
discipline’s  siurface  without  discussing  methodological  intricacies  since  a  thorough 
understanding  reqiiires  a  lengthy  study  of  volumous  books  and  journal  articles, 
many  of  which  discuss  only  one  technique  or  decision  making  method.  Some 
writers  suggest  that  "literally  thousands  of  models"  exist  (Souder  &  Mandakovic, 
1986:36).  Therefore,  Appendix  A  provides  a  cursory  examination  of  published 
materials  since  a  mastery  of  the  subject  requires  more  resources  than  this  docu¬ 
ment  affords.  Fahmi  and  Spatig’s  (1990)  research  provides  a  deeper  discussion  of 
the  subject  and  provides  excellent  guidance  to  managers  making  R&D  model 
choices.  Their  investigation  includes  a  binary  decision  tree  procedme  that  leads  to 
one  of  the  "twelve  archetypal  groups  of  methods"  for  R&D  project  selection  (Fahmi 
and  Spatig,  1990).  In  other  words,  it  helps  managers  pick  a  good  technique. 
Additionally,  a  Naval  Post  Graduate  thesis  by  Jordan  (1992)  furnishes  an  excellent 
overview  of  decision  making  methodologies  of  interest  to  the  military  manager. 
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2^.a  Is  Prolect  Selection  Tcchninue  an  Issue?  Under¬ 


standably,  R&D  decision  makers  must  select  projects  to  champion,  cancel  or  delay. 
Often,  individuals  or  agencies  make  these  decisions  through  procedures  used  to 
allocate  research  money.  A  report  requested  by  the  US  House  of  Representatives 
Committee  on  Science,  Space,  and  Technology  compared  funding  allocation 
methods  used  in  federal  research  agencies.  The  Office  of  Technical  Assessment 
(OTA),  responding  to  a  Congressional  request,  published  the  results  of  more  than 
125  interviews  conducted  vrith  managers  at  all  levels  in  its  report.  Federally 
Funded  Research:  Decisions  for  a  Decade  (Robinson,  1991:575-577).  The  report 
concludes  that 

. . .  setting  priorities  can  help  to  allocate  federal  resources  both  when  they  are  plentifuly  as 
they  were  in  the  1960%  and  when  they  are  scarce,  as  is  expected  tiirough  the  early  1990% 

(Chubin  A  Robinson,  1991:81-2) 

This  OTA  document  not  only  confirms  high  level  bureaucratic  interest  in  decision 
making  methodologies,  but  it  begs  the  question,  'is  something  wrong  with  the 
current  system?"  Indeed,  some  government  executives  and  at  least  one  OTA 
analyst  have  questioned  the  current  decision  making  process. 

Top  policy  makers  in  Congress  and  the  Executive  Branch,  as  welt  as  officials  at 
research  agencies  and  scientists  themselves,  have  acknowledged  that  the  nation  would  benefit 
from  clearer,  more  deliberate  selection  of  where  and  how  research  dollars  are  spent 
Preserving  the  plurulistic,  decentralized  system  of  federal  support  for  research  that  has  served 
well  for  40  years  now  requires  better  structures  for  making  choices  within  tiiat  system. 

(Chubin  &  Robinson,  1991:81-2) 


Many  documents  reveal  BoD’s  interest  in  the  subject.  Prior  to  the  OTA  report, 
research  project  selection  processes  served  as  the  focus  for  many  Depeutment  of 
Defense  (DoD)  initiatives.  The  abimdance  of  graduate  research  alone  confirms  this 
premise  (Brooks,  1979;  Prince,  1985;  Congdon,  1988;  and  Jordan,  1992)  and 
suggests  BoD’s  sustained  interest  in  R&D  resource  allocation  as  well. 
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2.3.b  Nature  and  Form  of  Techniques.  Humans  suffer  from 

many  constraints  that  preclude  consistency  in  good  decision  making.  One  of  these 
constraints,  bounded  rationality,  rea^piizes  our  finite  intellectual  capacities. 

Formal  decision  making  models  reduce  the  effects  from  this  constraint,  and  a 
myriad  of  other  potential  fallacies,  by  providing  techniques  for  structuring  the 
decision  problem,  clarifying  the  alternatives,  and  logically  formatting  the  selection 
process.  While  the  procedures  do  not  replace  a  decision  maker,  they  certainly 
enhance  an  individual’s  or  organization’s  decision  making  stability  and  continuity. 
They  also  help  safeguard  the  decision’s  relevant  factors  fimm  improper  weighting  or 
other  bias.  Jordan  notes  eight  potential  mitigation  factors  in  the  decision  process: 

/.  FUtering  bias 

2.  Experience  bias 

3.  Giving  priority  to  information  by  order  received 

4.  Memory  capacity  limitation 

5.  Conformity  to  prior  beliefs 

6.  Causal  chains  of  likely  outcomes  limitation 

7.  Risk  preference  bias 

8.  Independence  of  the  future  from  the  present 

(Jordan,  1992:29) 


Assuming  the  need  for  an  appropriate  decision  making  technique,  how  does 
one  explore  the  subject?  Do  literary  sources  support  a  hierarchy  subdivision  of  the 
methodologies?  Several  authors  have  approached  this  exploration  impasse  and, 
unfortunately,  they  have  chosen  different  paths,  which,  perhaps,  reflect  their 
unique  perspectives.  With  little  doubt,  the  ubiquitous  proliferation  of  classifi¬ 
cations  for  decision  making  techniques  reflects  the  general  abundance  of  methodol¬ 
ogies  and  conformds  the  exploration  issue.  In  the  following  two  examples  note  that 
the  first  author  divided  the  subject  into  four  groupings  while  the  second  author 
chose  two  different  categories. 
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It  is  useful  to  elass^  the  population  of  project  selection  models  as  classical 
methods,  portfo^  models,  project  evaluation  techniques  and  organizational  decision 
methods.  (Souder  and  Mandakovic,  1986:36) 

The  methods  proposed  in  Uterature  for  project  evaluation  and  selection  can  be 
classified  into  two  categories.  One  is  the  class  of  compensatory  models,  which  reduce  a 
multi-dimensk>nal  evabiation  to  a  singk-dimensionol  one  through  an  aggregating  ''value" 
function,  thereby  estabtishu^  tradeoffs  between  criteria.  Cost/benefit  analysis.  Multi  Attri¬ 
bute  Utility  Theory  (MAUT)  and  Analytical  Hierarchical  Process  (AHP)  can  be  cited  as 
examples  for  this  category _ 

The  other  category  of  methods  for  R&D  project  evaluation  and  selection  is  tiutt  of 
noncompensatory  models,  where  tradeoffs  between  attributes  ere  restricted.  One  group 
consists  of  Multiple-Criteria  Decision  Methods  of  the  ELECTRE  family . . . 

(Oral,  Kettani,  and  Lang,  1991:871-2) 


Fortunately,  for  the  purposes  of  this  thesis,  it  is  unnecessary  to  determine  or 
champion  a  particular  categorization  or  grouping.  It  is  critically  important, 
however,  that  the  reader  understands  that  diversities  of  techniques  exist.  The 
techniques  often  xxse  fundamentally  different  procedures  that  are  premised  upon 
widely  different  philosophies.  Congdon  (1988),  referencing  the  works  of  Stephen 
Cooley,  et  al.  (1986),  and  John  Gibson  (1981),  proposed  a  grouping  alternative 
different  from  the  two  prior  references.  The  format  provides  a  convenient  spring¬ 
board  for  a  cursory  examination  of  the  subject  and  with  only  minor  modification, 
serves  as  the  pattern  for  Appendix  A. 

1.  Scorify  models 

2.  Economic  models 

3.  Constrained  optimization  models 

4.  Decision  theory  models 

(Congdon,  1988:30) 


2,4  A  Historical  Perspective 

This  section  provides  an  appraisal  of  three  prior  AFIT  theses  on  R&D 
project  selection  methods  at  Wright  Laboratory,  formally  Imown  as  Air  Force 
Wright  Aeronautical  Laboratories.  These  documents  record  the  Air  Force’s 
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siistained  interest  in  R&D  decision  making  processes  and  point  toward  an  overall 
DoD  interest.  Their  material  span  the  decades  of  the  1970’s  to  the  1990’s.  The 
documents  are  presented  in  chronological  order  b^inning  with  the  first  thesis  by 
Air  Force  Captain  Terry  L.  Brooks  titled  Policy  Capturing  of  Management  Personnel 
through  Project-Selection  Decision  Making  in  an  Air  Force  Research  and  Development  Laboratory 
(Brooks,  1979).  A  second  selection,  by  Jeremy  R.  Prince,  Research  and  Development  Pro¬ 
ject  Selection  Methods  at  the  Air  Force  Wright  Aeronautical  LaboratorieSy  addresses  how  Air 
Force  managers  select  research  projects  to  create  a  laboratory  portfolio  (Prince, 
1985).  The  third  thesis.  Factors  Affecting  the  Adoption  of  R&D  Project  Selection  Techniques  at 
the  Air  Force  Wright  Aeronautical  Laboratories,  authored  by  Jonathan  D.  Congdon 
expands  on  Prince’s  research  and  presents  the  Air  Force  Wright  Aeronautical  Labo¬ 
ratory  managers’  attitudes,  perceptions,  and  adoption  affecting  variables  for 
research  portfolio  selection  (Congdon,  1988). 


2.4«a  Brooks  Stlldy»  in  the  mid  to  late  197()’s  Doctor’s  Michael  J. 

Stahl  and  Adrian  M.  Harrell  of  AFIT  conducted  extensive  interviews  within  Wright 
Laboratory.  They  successfully  isolated  six  factors  that  laboratory  personnel 
contends  were  used  during  R&D  project  selection  (i.e.  the  factors  laboratory 
decision  makers  judged  as  critical).  They  were 

1.  Cost-Benefit  Ratio 

2.  Technical  Merit,  characterized  as  new  or  better  capabilities 

3.  Resource  AvaUaMSbf,  the  availability  of  personnel,  equipment,  or 
facilities 

4.  Likelihood  of  Success,  the  probability  of  achieving  technical  success 
within  the  time  constraints 

5.  Time  Period,  based  upon  the  estimated  project  completion  time 

6.  Air  Force  Need,  or  the  degree  that  the  Air  Force  had  articulated  a  need. 

(Brooks,  1979:87) 
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Captain  Brooks  employed  this  information  in  a  policy  capturing  instrument 
and  then  surveyed  laboratory  dedsim  makers,  partitioning  them  level,  division, 
and  type  of  laboratory  project.  His  goal  was  to  model  individual  laboratory 
members  and  determine  if  a  consensus  on  decision  making  criteria  existed  among 
management  levels  and  within  laboratory  directorates.  Brooks  tested  eight  specific 
hypothesis  and  foimd  that  Wright  Laboratory  managers  indeed  used  all  six  factors 
identided  by  Stahl  and  Harrell  (Brooks,  1979:89).  Additionally,  he  determined 
that: 

. . .  there  was  not  a  consensus  in  the  decision  nuking  process  among  managers  of  the 
management  levels  of  the  63  projects,  or  the  managers  in  the  divisions  of  the  6J  and  63 
ejects.  Also,  the  managers  of  both  projects  tUd  not  use  a  decision  making  process  exacdy 
the  same  as  that  which  they  perceived.  (Brooks,  1979:89) 

Brooks’  findings  suggest  that  Wright  Laboratory  managers  could  not  correctly 
articulate  the  modeKs)  they  actually  used. 

2.4.b  Prince  Study.  Prince’s  research  focused  on  exactly  how 

Wright  Laboratory  managers  selected  their  research  projects.  He  iised  a  personal 
and  telephone  interview  technique  to  fulfill  ten  specific  objectives.  His  research 
provides  the  first  historical  data  base  on  tlie  seven  technical  constituent  labs  of 
Wright  Laboratory  known  today  as  Directorates.  Prince’s  study  included  demo¬ 
graphic  data  on  the  managers  surveyed.  His  findings  detail  formal  R&D  decision 
method  awareness,  technique  use,  and  the  laboratory’s  desire  to  change  project 
selection  methodology.  He  also  recorded  laboratory  decision  maker  ratings  on  the 
importance  of  various  research  project  selection  factors. 

Prince  concluded  that  the  "top  three  factors"  in  project  selection  were:  Air 
Force  need  or  organizational  goals,  technical  merit,  and  resource  availability 
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(Prince,  1985:37).  He  noted  that  only  30  percent  of  the  respondents  int^riewed 
used  formal  decision  making  methods.  This  may  appear  as  a  surprisingly  small 
fraction  since  the  remaining  70  percent  of  the  interviewed  respondents  made  their 
decisions  without  the  help  of  formal  decision  making  tools. 

Prince  also  identified  a  laboratory  wide  phenomena  that  formal  decision 
making  formal  methods,  when  employed,  were  "relatively  simple  to  learn  and  easy 
to  use"  (Pnnce,  1985:38).  Although  many  respondents  were  aware  of  formal 
decision  making  techniques,  usage  was  low.  Apparently,  the  only  exceptions  were 
cases  with  relatively  large  budgets.  Prince  attributed  this  behavior  to  a  feeling  ex¬ 
pressed  by  many  respondents  that  "mc^t  of  the  decision  making  tools  are  impracti¬ 
cal  because  of  disharmony  with  the  existing  laboratory  management  style  and  the 
technical  weakness  of  the  methods"  (Prince,  1985:38). 

Further,  Prince  found  low  positive  response  to  any  initiative  to  change 
decision  making  methods  in  the  laboratories,  a  finding  that  dramatically  changed 
in  the  years  between  Prince  and  Congdon’s  research.  Prince  noted  that  formal 
training  had  httle  impact  on  decision  making  methodology  and  deduced  that  formal 
methods  were  used  by  upper  management  because  they  were  "under  greater 
scrutiny  than  managers  at  lower  levels"  (Prince,  1985:39).  He  also  found  that  the 
use  of  formal  decision  methods  escalated  with  the  dollar  values  of  the  projects  or 
portfolios  involved. 

2.4.C  Congdon  Stildv*  Congdon  validated  many  of  Prince’s  find¬ 
ings.  His  thesis  pursued  three  research  objective’s  which  roughly  paralleled 
Prince’s  objectives.  Specifically  Congdon  wanted  to  know: 
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1.  TIu  meikads  uud  by  managers  to  seket  RAD  projects  and  oBoetUe  resources; 

2.  The  atdttkks  and  perceptions  of  managers  toward  the  use  offwmal  RAD 
project  selection  techniques;  and 

3.  The  fsetors  qffecting  the  adoption  of  these  techniques  by  RAD  managers. 

(Congdou,  1988:35) 


Congdon  surveyed  43  Wright  Laboratory  managers,  all  occupying  the 
position  of  Deputy  Division  Chief  or  higher.  He  selected  twelve  managers,  three 
from  each  directorate,  for  intensive  follow-up  interviews.  Interestingly,  Congdon’s 
research,  occurring  three  years  after  Prince’s,  established  that  the  survey  respon¬ 
dents  "in  general,  are  not  familiar  with  most  R&D  project  selection  techniques” 
(Congdon,  1988:61).  He  confirmed  Prince’s  earlier  observation  that  formal  R&D 
project  selection  technique  use  was  limited,  although  his  respondents  expressed  a 
desire  to  learn  more  about  the  different  techniques.  Congdon  noted  that  Material 
Laboratory’s  (now  known  as  Material  Directorate)  project  selection  process  tended 
to  be  "somewhat  more  structiured"  (Congdon,  1988:69). 

Congdon  documented  an  Air  Force  environmental  effect  he  titled  "require¬ 
ments  pull"  which  motivated  managers  to  demonstrate  R&D  benefits  of  a  specific 
program  in  economic  terms.  Additionally,  he  writes, ". . .  respondents  indicated 
that  budget  constraints  force  managers  to  select  those  R&D  projects  which  are  less 
risky,  and  provide  immediate  short-term  benefits”  (Congdon,  1988:71). 

Congdon  also  noted  little  likelihood  that  any  existing  formal  technique  could 
adequately  consider  all  relevant  Wright  Laboratory  decision  making  criteria  due  to 
the  following: 

1.  The  input  data  required  by  formal  techniques  would  be  t^icult  to  develop 

2.  The  matitematics  involved  with  the  quan^tcation  of  project  selection  decisions  was 
more  complex  than  necessary 

3.  The  outputs  from  the  potential  techniques  would  yield  irrelevant  information. 

(Congdon,  1988:76) 
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Periiaps  Coagdon’s  dbeervations  explain  the  disparity  between  decision  makers’ 
willingness  to  change  procedures  and  documented  lack  of  change. 


2.5  Lateral  Airfftil  Tft<*hnologv 

This  section  provides  a  designer's  glimpse  of  tluree  applications  for  lateral 
airfoil  technology.  Lateral  airfoils  typify  a  technology  that  the  Air  Force  could 
research.  Bibliometric  evidence  hum  United  States  patents  granted  for  specific 
applications  on  aerospace  vehicles  exist  and  serves  as  the  springboard  for  discus¬ 
sion  of  the  concept  and  its  use  in  this  study. 

The  lateral  airfoil  patent  designs  shown  in  Appendix  B  represent  what 
many  inventors  considered  as  a  truly  remarkable  discovery  or  innovation  for  their 
time.  The  patents  captured  both  the  designers’  dreams  and  best  understanding  of 
the  concepts.  A  summary  presented  in  Appendix  B  acknowledges  the  designers’ 
contributions  to  lateral  airfoil  understanding  and  establishes  a  trend  that  offers 
unique  operational  characteristics  and  enhanced  capabilities  for  aerospace  flight. 
However,  it  is  important  to  note  that  with  aerospace  vehicles,  a  patent  does  not 
establish  airworthiness. 

2.5.a  AerOSDiM^ft  Vehicle.  An  easy  way  to  explain  lateral  airfoils 

is  to  compare  them  to  known  and  well  understood  technology.  Today’s  helicopter 
serves  this  roll  since  most  people  have  witnessed  the  helicopter’s  unique  flight 
characteristics  like  hover,  vertical  takeoff,  and  vertical  landing.  During  these 
maneuvers  the  helicopter  generates  lift  ifrom  its  main  rotor  blades  to  overcome  the 
other  forces  acting  on  the  vehicle,  like  gravity.  If  one  considers  a  helicopter  sitting 
on  the  ground,  as  the  main  rotor  blades  spin  faster  and  faster  while  the  blade  pitch 


is  increased,  the  vehicle  slowly  rises  vertically  into  the  air.  likewise,  if  the  pilot 
decreases  the  main  rotor  blade  pitch  or  allows  the  blade  to  slow,  then  the  vehicle 
settles  gently  to  the  ground.  The  main  rotor  blade  spinning  about  the  vertical  axis 
of  the  vehicle,  makes  all  this  happen. 

A  lateral  airfoil  produces  the  same  forces  as  a  helicopter  main  rotor  blade. 
The  only  difference  is  in  the  axis  of  movement.  Lateral  airfoils  revolve  around  the 


Figure  5.  Lateral  Airfoil  Aerospace  Vehicle  Design 

vehicle’s  horizontal  axis,  while  the  helicopter’s  main  rotor  blades  rotate  about  the 
vertical  axis.  The  aerodynamics  follows  all  known  laws.  A  deeper  discussion  of  the 
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flight  fivoes,  control  mechanisms,  and  capabilities  of  the  lateral  airfoil  aerospace 
vehicle  appears  in  Appendix  C. 

The  artist’s  conception  in  Figure  5  presents  a  classic  military  fighter  design 
with  landing  gear  retracted.  Notice  that  the  design  uses  more  than  one  blade  just 
as  helicopters  usually  use  more  than  one  blade  on  top.  Also,  the  aerospace  vehicle 
exploits  more  than  one  set  of  blades.  This  allows  better  roll  axis  control  because 
the  blade  sets  counter-rotate  and  eliminate  torque. 

Helicopters  produce  much  air  movement.  This  is  most  noticeable  near  the 
ground  by  the  characteristic  main  rotor  blade  down-wash.  likewise,  the  lateral 
airfoil  produces  a  downwash  effect,  which  if  left  unducted  would  build  inside  the 
vehicle’s  fuselage.  The  artist  concept  ducts  the  down-wash  out  the  back  of  the  craft 
producing  a  forward  thrust  component. 

There  are  many  novel  lateral  airfoil  designs  documented  with  the  federal 
government.  Every  attempt  has  been  given  to  exhaustively  survey  all  pertinent 
United  States  Patents  and  present  them  in  Appendix  B,  but  one  or  more  patents 
may  have  been  overlooked  that  describe  or  significantly  enhance  knowledge  on 
lateral  airfoils.  The  patents  and  descriptions  presented  in  the  appendix  were  gath¬ 
ered  through  a  patent  search  commissioned  by  the  author  in  1978  and  updated 
through  a  LEXIS  data  base  search  using  the  key  phrases;  (rotat!  or  mov!)+(wing  or 
airfoil)+(lateial  w/5  airfoil). 

2^.b  Gas  Turbine  Engine.  For  the  purposes  of  this  discussion, 

one  may  characterize  the  gas  turbine  engine’s  operation  in  basic  terms.  An  axial 
flow  engine  draws  firesh  air  in  the  front  or  intake,  pressurizes  it  through  successive 
stages  of  ccanpression,  mixes  fuel  with  the  highly  compressed  air,  ignites  the 
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mixture,  uses  a  turfame  to  extract  some  portiou  of  the  energy  from  the  air  to  power 
the  compressor,  and  blasts  any  super  hot  air  remaining  out  the  anginAa  aft  The 
engine  gets  its  name  firom  the  idea  that  the  air  flows  in  a  straight  or  aTial  faahinn 
from  intake  to  exhaust 


Figure  6  shows  a  lateral  airfoil  gas  turfaine  engine  anKka  any  known  Haaign 
Yet,  it  shares  much  in  common  with  the  'traditional*  arial  flow  counterpart  Recall 
that  the  lateral  airfoil  aerospace  vehicle  characterized  in  section  2.5.a  suffered  from 
a  down*wash  phenomena  where  air  pressure  built  up  inside  the  vehicle’s  fuselage. 
One  might  characterize  this  as  a  vice  since,  if  left  unchecked,  it  would  destroy  the 
vehicle.  However,  with  gas  turbine  engine  applications  this  vice  becomes  a  virtue. 
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Figure  6  also  shows  the  intake  air  coming  into  the  engine  fiom  the  sides. 
Air  enters  firom  all  360  degrees.  The  lateral  airfoils  perform  the  same  function  as 
compressor  blades.  As  the  blades  revolve  aroxind  the  engine’s  longitudinal  axis 
they  draw  air  down  deeper  into  the  core.  This  looks  like  a  <7clone  effect  to  a 
casual  observer  and  may  serve  as  an  appropriate  name  for  the  engine.  Just  as 
with  the  gas  t\irbine  engine,  there  may  be  more  than  one  stage  of  compression. 

The  lateral  airfoil  compressor  and  turbine  blades  have  advantages  over 
their  counterparts  in  the  conventional  axial  flow  engine.  These  benefits  stem  firom 
the  fact  that  the  blades  are  supported  on  both  ends.  First,  one  can  make  longer 
blades  to  p^lll  in  larger  volumes  of  air.  Secondly,  if  the  volume  of  air  required 
remains  the  same,  than  the  blades  can  be  made  lighter  since  they  don’t  have  to 
carry  the  same  amount  of  stress  at  their  root.  Other  benefits  arise  as  well.  For 
example,  the  engine  can  produce  massive  amounts  of  thrust  without  requiring  a 
large  diameter  fix>ntal  area.  The  removal  of  the  large  flat  compressor  disk  reduces 
the  vehicle  air  drag.  Finally,  although  not  exhaustively,  the  engine  demonstrates 
better  low  observable  characteristics  since  the  big,  radar  reflecting,  compressor 
disk  becomes  a  longer,  thinner  axial  member. 

The  burner  section  of  the  lateral  airfoil  engine  may  look  similar  to  its 
counterpart  on  the  axial-flow  engine,  but  the  tiubine  section  makes  a  radical 
departure.  Lateral  airfoil  turbine  blades  revolve  around  the  engine’s  longitudinal 
axis  and  extract  heat  energy  firom  the  hot,  compressed  air  in  much  the  same 
fashion  as  a  conventional  turbine  blade.  The  fluid  passage  way  gradually  expands 
as  the  air  surfaces  firom  the  engine  core.  The  successive  stages  of  turbines  extract 
needed  energy  to  drive  the  lateral  airfoil  compressor  section  while  the  remaining 
energy  is  ducted  for  some  useful  purpose. 


One  may  visualize  a  compressor  drive  shaft  located  on  the  engine’s  exterior 
as  opposed  to  an  axial-flow  design  where  the  drive  shaft  sits  in  the  engine’s  core. 
This  also  presents  some  unique  characteristics  since  the  shaft  can  be  spht  into 
more  than  one,  but  what  is  more  important,  the  engine  centerline  section  is  now 
free  for  other  uses.  For  example,  one  could  employ  a  lateral  airfoil  compressor  and 
turbine  section  on  the  engine’s  outer  surface  area  and  place  a  ram  jet  in  the  center. 
This,  of  course,  would  offer  unique  combiimd  cycle  propulsion  capabilities.  Alterna¬ 
tively,  one  corild  use  an  axial  flow  compressor  or  tiirbine  sections  with  an  appropri¬ 
ate  lateral  airflow  complement  to  produce  still  more  radical  design  combinations 
each  with  its  own  imique  capabilities. 

One  can  see,  without  much  engineering  rigor,  that  lateral  airfoils  offer  unique 
capabilities. 


2.5.C  Naval  Propillsion.  Naval  technology  applications  lay  outside 

the  normal  research  work  of  Wright  Laboratory.  Yet,  the  lateral  airfoil  or,  more 
correctly,  the  lateral  hydrofoil  applications  follow  so  naturally  that  they  merit  a 
cursory  mention  here. 

Lateral  hydrofoil  flmd  dynamic  appUcations  offer  a  new  dimension  for 
technology  employment.  A  concept  drawing  of  the  lateral  hydrofoil  dynamic 
propulsion  appears  in  Figure  7.  The  foil’s  characteristics  parallel  the  Voith- 
Schnider  propeller  designs  common  in  marine  appHcations.  In  fact,  one  may 
envision  the  lateral  hydrofoil  as  a  Voith-Schnider  propeller  rotated  90  degrees. 

The  lateral  hydrofoil  can  produce  many  of  the  same  thrust  vectors  found  in  a 
Voith-Schnider  blade,  especially  if  the  blades  are  helixes.  However,  there  are  some 
distinct  differences. 
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One  noticeable  diSerence  between  the  Voith'Scbnider  propeller  and  the 
lateral  hydrofoil  is  the  placement  of  the  foil.  The  lateral  hydrofoil  can  be  partially 
or  fully  STibmerged  and  it  yields  a  thrust  ^^ctor  created  from  the  fluid  buildup  in 
the  center  of  the  revolving  blades.  Further,  the  lateral  hydrofoil,  if  foshioned  with 
substantial  blade^oil  widths  and  helix,  can  support  the  vehicle  while  propelling  it 
at  high  speed. 

2.6  Chapter  Smmnarv 

The  literature  review  began  with  a  discussion  of  Wright  Laboratory, 
including  its  organizational  structure,  mission,  and  demographics.  A  survey  of 
R&D  project  selection  techniques  followed.  The  nature  and  form  of  the  techniques 
were  briefly  sketched  while  the  reader  was  referred  to  Appendix  A  for  a  deeper 
discussion,  including  the  strengths  and  weaknesses  of  each  category  by  method- 
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ol(^.  Next,  past  research  on  Wright  Laboratory  with  material  focused  on  three 
AFIT  research  theses  firom  1979  through  1988  was  presented.  Finally,  the  chapter 
ended  with  a  comprehensive  overview  of  lateral  airfoil  technology.  The  discussion 
included  an  aerospace  vehicle,  gas  turbine  engine  and  naval  propulsion  applica¬ 
tions.  Again,  deeper  discussions  of  the  topic  appear  in  the  Appendix  A.  The  next 
chapter  explores  the  research  methodologies  used  to  secure  the  answers  for  the 
questions  generated  from  the  specific  issue  identified  in  chapter  I. 
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III.  Methodolotfv 


3.1  Introduction 

This  chapter  builds  from  the  research  questions  and  the  prior  studies 
discussed  in  Chapters  I  and  n.  It  explains  how  lateral  airfoils  were  used  to  gather 
information  on  Wright  Laboratory’s  decision  making  criteria.  The  following  para¬ 
graphs  recormt  the  method.ology  iised  to  explore  the  specific  issue  while  meeting 
thii  research  objectives  and  answering  the  research  questions.  Generally,  the 
research  methodology  synthesizes  answers  for  the  investigative  research  questions 
through  three  techniques,  surveys,  a  patent  search,  and  interviews.  The  format  for 
discussing  the  methodology  follows  the  investigative  questions  as  listed  in  Chapter 
I.  Each  of  the  investigative  questions  highlighted  a  xmique  facet  of  the  research 
objectives  and  used  a  tailored  methodology. 

3.2  Question  1;  Proiect  Selection  Methods 

What  methods  do  managers  use  to  select  R&D  projects  and  allocate  laboratory 
resources  at  Wright  Laboratories? 

Previous  research  by  Prince  (1985)  and  Congdon  (1988)  offers  both  a 
historical  basis  for  iinderstanding  the  decision  making  process  and  a  variety  of 
research  tools  to  explore  this  question.  Further,  their  research  results  contribute  a 
benchmark  with  which  to  measure  Wright  Laboratory  decision  maker  change, 
retrenchment,  or  progress  with  formalized  decision  making  techniques.  This 
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research  determines  whether  the  laboratory  has  enhanced  older  published  tech¬ 
niques  or  initiated  new  methodologies  in  recent  years. 

Both  Prince  and  Congdon  collected  demographic  data  and  respondent 
attitude  data  on  Wright  Laboratory’s  managerial  preference  on  many  formal 
project  selection  techniques  (Congdon,  1988:38).  Their  results  define  the  parame¬ 
ters  for  the  historical  data  base  that  will  be  scrutinized  in  conjunction  with  the 
results  of  this  study. 

Congdon’s  research  text  included  his  copy  of  a  choice  survey  questionnaire. 
He  adapted  his  survey  instrument  fi’om  a  "Likert-based  instrument  provided  by  Dr. 
William  Souder,  a  professor  at  the  University  of  Kttsburgh"  (Congdon,  1988:4). 
Congdon  states  that,  "because  the  instnnnent  was  developed  by  an  expert  in  the 
field  of  implementation  research,  its  internal  validity  was  not  questioned” 
(Congdon,  1988:37).  However,  experts  in  Likert  based  survey  formats  may  find 
some  fault  with  specifics  of  Congdon’s  instrument.  Despite  obvious  limitations, 
this  research  uses  Congdou’s  instrument.  Re-administrating  Congdon’s  survey 
instrument  in  this  thesis  facilitates  many  concurrent  and  historical  anal3rsis 
options.  Additionally,  it  creates  a  consistent  research  siurvey  data  base,  albeit  with 
unknown  instrument  validity,  for  fiitiue  research.  The  fact  that  approximately 
four  years  elapse  between  research  on  this  aspect  of  Wright  Laboratory  has  not 
been  overlooked. 

Prince,  the  first  researcher  delving  into  Wright  Laboratory,  used  an  inter¬ 
view  questionnaire  with  many  of  the  same  questions  asked  later  by  Congdon 
(Prince,  1985:43-55;  Congdon,  1988:93-98).  A  cursory  inspection  reveals  that, 
despite  many  similarities,  Congdon  employed  a  much  richer  survey  format  asking 
more  in-depth  questions  than  Prince.  Congdon’s  form  fills  three  sections  and  spans 
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eleven  pages  (Congdon,  1988:93-98).  Prince’s  interview  format  rests  on  three 
pages  (Prince,  1985:43-45).  While  Congdon’s  survey  instrument  was  chosen  as  a 
more  appropriate  information  gathering  device,  note  that  some  minor  changes  were 
needed  to  reflect  the  widespread  organizational  restructiiring  and  renaming  that 
has  affected  Wri^t  Laboratory  since  1988.  A  copy  of  the  1993  survey  appears  in 
Appendix  D. 

The  specific  methodologies  used  in  this  research  effort  differ  from  both 
Congdon  and  Price.  The  1993  survey  mailing  list  includes  a  representative  sample 
of  Wright  Laboratory  Directorates.  Each  branch  received  two  surveys  based  upon 
the  Air  Force  internal  distribution  system.  A  short  discussion  here  may  prove 
useful  to  readers  unfamiliar  vdth  the  internal  military  mail  address  structure. 

The  Air  Force  uses  letters  to  represent  various  offices.  Usually,  more 
important  offices  in  an  organizational  hierarchy  have  fewer  letters  in  their  office 
symbols  or  mailing  addresses.  For  example,  a  Directorate  might  have  a  two  letter 
office  symbol  like  PQ.  Typically,  organizations  directly  below,  and  within  a 
directorate,  share  the  same  first  two  letters  plus  th^  own  unique  identifier.  So,  a 
division  might  be  PQR  in  this  example.  The  1993  survey  sample  was  stratified 
Tising  this  hierarchy  format.  One  sxirvey  went  to  each  Directorate  level,  two  letter 
office  s3mibol.  The  Division  levels,  with  three  letter  office  symbols,  also  received 
one  survey.  Lower  tier  offices,  with  four  or  more  letters,  all  received  two  surveys 
each.  In  total,  Wright  Laboratory  received  three  himdred  thirteen  surveys  or  one 
sxirvey  per  ten  people. 

3,2.a  Survey  Instrument  Part  1,  Survey  questions  one 
through  sixteen  (see  Appendix  D)  identify  decision  maker  familiarity  and  extent  of 
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usage  with  published  R&D  project  or  portfolio  selection  techniques  (Congdon, 
1988:93).  The  responses  to  these  questions  were  processed  with  the  SAS/STAT 
software  package.  The  package  produces  both  descriptive  statistics  and  variability 
information  on  the  survey  data.  Statistically  significant  response  variance  among 
the  Wright  Laboratory  Directorates  was  analyzed  using  a  Kruskal-Wallis  non- 
parametric  test  process  on  AFITs  Academic  Support  Computer.  This  Kruskal- 
Wallis  procedure  carries  no  restrictive  assumptions  of  respondent  data  ’normality’. 
Congdon  notes  that  the  Kruskal-Willis  test  is  an  appropriate  procedure  because  of 
"its  robustness  and  conservatism"  (Congdon,  1988:40).  Stratification  of  the  sample 
population  by  directorate  enables  sub-population  data  analysis  while  giving  greater 
procedural  flexibility  for  the  different  strata  (Emory,  1980:167).  However,  not  all 
directorates  were  included  in  this  procedtire.  The  study  was  limited  to  Plans  and 
Programming  and  the  seven  technology  directorates.  Results  from  the  data 
analysis  follow  in  thesis  Chapter  IV. 

Questions’  17  through  28  used  Likert-based  questions  to  "measure  the 
respondent’s  attitude  and  perception  toward  the  use  of  formal  project  selection 
techniques"  (Congdon,  1988:38).  Again,  SAS/STAT  provides  descriptive  analysis 
and  variability  information  on  the  data  where  appropriate.  The  analysis  employs  a 
Kruskal-Wallis  non-parametric  test  for  respondent  variability  between  the  director¬ 
ates.  However,  many  of  survey  questions  17-28  lay  beyond  the  domain  of  the  1993 
research  and  the  results  are  not  addressed  in  this  study. 

3.2.b  Survey  Instniment  Part  2.  Part  II  of  the  survey  seeks 

information  on  "management’s  willingness  to  adopt  formal  R&D  project  selection 
techniques"  (Congdon,  1988:8).  While  this  information  was  not  needed  for  the 
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study,  it  was  retained  to  preclude  unexpected  interaction  effects  resulting  from 
removing  this  survey  instrument  section.  An  edited  survey  instrument  might 
compromise  data  comparisons  with  Congdon’s  earlier  siurvey.  Part  II,  therefore, 
remained  in  the  1993  survey.  The  data  is  included  in  the  data  base  (see  Appendix 
E),  however,  an  analysis  of  the  data  is  left  for  future  study. 

3»2.C  Survey  Instruniftiit  Part  3.  This  section,  with  questions 

59  through  70,  addresses  demographic,  budgetary,  and  training  data  (Congdon, 
1988:97-98).  Again,  the  SAS/STAT  software  package  provides  both  descriptive  and 
/ariability  information.  One  cannot  compare  the  1993  resxilts  with  1985  or  1988 
without  first  vmderstanding  the  respondent  demographics. 

3,3  Question  2;  Lateral  Airfoil  ResearcL  Viability 

What  establishes  lateral  airfoils  as  a  potentially  viable  research  project! 

Narratives  and  drawings  presented  in  Chapter  II  detailed  some  of  many, 
potential  lateral  airfoil  applications.  In  addildon.  Appendix  A’s  patent  search 
shows  trends  with  the  design  through  the  years.  Together,  these  two  sections 
establish  the  research  potential  of  lateral  airfoil  technology.  Establishing  an 
answer  to  question  is  pivotal  to  the  investigation.  The  stakes  are  high  because,  if 
the  lateral  airfoil  project  appears  too  ridiculous,  then  it  becomes  unsuitable  as  a 
research  project  placebo.  This,  in  t\im,  compromises  the  entire  "criteria"  data 
collection  process  discussed  later  in  this  chapter. 

The  bibliometric  approach  looks  to  literature,  historical  records,  and  other 
media  sources  for  a  trend.  If  enough  bibliometric  evidence  surfaces,  than,  to  a 
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certain  degree,  it  established  lateral  airfoil  teclmolc^s  viability  for  institutional 
research.  Specifically,  if  a  long  term  trend  exists  showing  increased  understanding 
and  sustained  interest  in  aerospace  applications,  then  lateral  airfoils  may  hold 
some  research  potential  for  Wright  Laboratory. 

Patents  represent  one  body  of  bibliometric  evidence  for  investigation.  They 
are  ideal  mediums.  Gaining  access  to  the  United  States  patents  records  requires 
only  moderate  levels  of  effort.  Furthermore,  as  legal  documents,  they  carry  high 
credibility.  Additionally,  computer  search  services  like  LEXIS  that  maintain 
patent  data  bases,  grant  researchers  an  expanded  capability.  The  computer  can 
successfully  isolate  appropriate  patents  by  using  a  "key  word"  search  process,  llie 
data  collection  effort  includes  a  LEXIS  key  word  search  using  the  follovnng  format: 
(rotat!  or  mov!>+(wing  or  airfoil>f(lateral  w/5  airfoil).  Results  firom  the  quest 
appear  in  Appendix  A. 

A  validation  test  for  bibliometrics’  adeptness  at  predicting  research  potential 
was  included  with  the  interviews  described  later.  Each  interview  participant 
gauged  the  potential  of  lateral  airfoil  technology.  They  were  asked  if  they  would 
support,  oppose,  or  table  the  placebo  project.  The  responses,  processed  with  the 
SAS/STAT  package,  appear  in  Chapter  IV. 

3.4  Question  3;  Attributes  Used  in  Portfolio  Selection 

'What  determinate  attributes  do  decision  makers  use  in  the  final  choice  set 
during  research  program  portfolio  selection? 

Compared  with  Prince  or  Congdon’s  prior  research,  this  question  explores  a 
distinctly  deeper  stratum  in  the  decision  making  process.  It  sets  the  stage  for 
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subsequent  modeling  of  the  Wright  Laboratory  decision  process.  'Hie  criteria 
gathering  methodology  must,  therefore,  provide  credible  determinant  attributes 
that  can  fit  into  a  model.  Techniques  available  for  building  a  decision  model  vary, 
but  include  the  methodology  employed  by  Brooks  (Brooks,  1979)  and  Metric 
Conjoint  Analysis  as  described  by  Louviere  (Louviere,  1988).  In  an  efibrt  to 
maintain  some  consistency,  the  1993  data  collection  method  uses  a  typical  metric 
coiyoint  analysis,  opinion  measurement  tool. 

The  exploratory  natiire  of  the  determinant  attribute  identification  process 
suggests  a  personal  interview  format  for  the  data  collection  method  (Emory  and 
Cooper,  1991:146).  Indeed,  one  prior  Air  Force  research  thesis  documents  the 
results  of  interviews  on  this  subject  conducted  by  Doctors  Stahl  and  Harrell  of  the 
Air  Force  Institute  of  Technology  (Brooks,  1979:89).  Unfortunately,  Brooks’  work 
fails  to  describe  how  their  criteria  were  distilled.  Since  an  accurate  decision 
making  model  presupposes  valid  determinant  attributes,  the  1993  research  targets 
this  material  void. 

An  interview  format  affords  the  best  opportunity  to  explore  the  issue,  but 
researcher  time  proved  a  limiting  factor  at  AFIT.  The  1993  interview  schedule 
hinged  on  securing  the  Wright  Laboratory  sample  within  a  constrained  period  of 
three  weeks.  Cost,  in  dollars,  for  data  collection  presented  no  problem,  but  the 
time  span  allocated  for  the  interviews  had  to  fit  into  fifteen  work  days.  A  dispro¬ 
portionate  stratified  sampling  technique  was  chosen  (Emory  1980:168-169). 
Throughout  the  period,  thirty-six  interviews  were  conducted  for  a  sampling  rate  of 
one  percent  of  the  popxilation. 

Emory  cautions  that  a  small  sample  size  based  on  convenience  has  little 
status  (Emory  1980:177).  But,  he  rejects,  as  folklore,  the  belief  that  sample  size 
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must  bear  some  proportion  (ie.,  10  percent)  to  total  population  size  (Emory 
1980:153).  In  &Gt,  Emory  notes  that  simi^  stratification  may  be  mcne  efBdent 
than  random  sampling  with  disproportionate  sampling  yielding  even  greater 
efficiencies.  The  interview  schedule  employed  a  disproportionate  sample  of  Wri^t 
Laboratory  constituents  using  approximately  six  interviews  per  directorate.  Some 
support  directorates  were  excluded  fix)m  the  interviews  because  they  lacked  a 
research  thrust.  The  interview  schedule  also  considered  disparate  organization 
within  the  lower  organizational  levels.  For  example,  some  directorates  possess  no 
section  levels  within  their  entire  organization.  Within  directorates  that  have 
section  levels,  all  branch  levels  may  not  have  equal  numbers  of  sections  under 
their  authority.  Finally,  because  the  directmrates  used  to  be  independent  laborato¬ 
ries  prior  to  the  creation  of  ASC’s  four  "super  laboratories,”  not  all  branches 
perform  equal  management  roles.  Therefore,  the  agenda  included  interviews  with 
two  participants  each  at  levels  of  directorate  or  division,  branch,  and  section  for 
Wright-Patterson  Air  Force  Base  as  appropriate. 

The  typical  interview  format  includes  a  brief  introduction  for  the  inter¬ 
viewer  and  participant  which  usually  last  one  to  two  minutes.  Following  the 
introductions,  each  participant  received  a  five  minute  cursory  overview  of  the 
lateral  airfoil  placebo  project  Since  lateral  airfoils’  proffer  a  radical  appearance, 
they  serve  as  an  easy  medium  for  discussion.  Further,  they  are  so  xinknown  that 
participants  approach  the  technology  at  roughly  the  same  level  of  innocence.  As 
the  interview  progresses,  the  determinant  attributes  articulated  by  the  participant 
are  dociunented  on  the  bottom  margin  of  the  interview  guide  response  sheet  that  is 
included  in  Appendix  F.  For  instance,  if  the  decision  maker  questions  the  design 
cost  or  technical  merit,  then  those  two  determinant  attributes  were  recorded  on  the 
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int^view  guide  along  with  all  other  discrimination  parameters.  Please  note  that 
the  interviewer  does  the  writing  during  this  "brain  storming"  sessicm. 

However,  after  the  participant  exhausted  his/her  personal  set  of  determi¬ 
nant  attributes,  then  they  were  asked  to  review  a  draft  list  of  attributes  recorded 
on  the  interview  guide.  From  this  point  forward,  the  participant  makes  all  neces¬ 
sary  marks  on  the  interview  guide.  The  participants  are  first  asked  to  record  the 
draft  criteria  in  their  own  words.  Participants  are  encouraged  to  expand  or  delete 
criteria  from  their  "brain  storming"  list  as  appropriate.  Then  participants  marked 
their  opinion  on  the  utility  of  each  of  their  final  criteria.  A  one  hundred-fifty-milli¬ 
meter  line  mark  scale,  common  with  metric  conjoint  studies,  serves  as  the  mea- 
siuement  devise. 

Emory  and  Co<^r  caution  against  potential  interview  problems.  "In 
personal  interviews,  the  researcher  must  deal  with  two  mayor  problems:  bias  and 
cost"  (Emory,  1991:327).  A  need  for  unbiased  information  requires  that  the  inter¬ 
viewer  not  alter  the  questions  or  otherwise  influence  the  interview  subject.  An 
interview  guide  furnishes  the  best  chance  for  minimizing  this  type  of  error.  The 
constrained  time  limit  focused  the  scope  of  this  research  on  subjects  gec^raphically 
near  Wright-Patterson  Air  Force  Base.  This  constraint,  however,  virtually  elimi¬ 
nates  cost  as  a  factor.  Bias,  then,  remains  as  the  only  potential  interview  problem. 

Characteristically,  bias  can  occur  firom  three  factors:  sampling  error, 
response  error,  and  nonresponse  error.  Regrettably,  some  sampling  error  surfaces 
in  the  1993  interviews  since  the  sample  population  of  Wright  Laboratory  decision 
makers  may  not  truly  represent  the  normal  population  qualities  for  decision 
makers  fiom  the  past,  present,  or  fiitiure. 
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Nonresponse  error  proves  negligible  since  the  interviews  were  scheduled 
throughout  a  three-week  time  span  allowing  more  than  one  interview  opportunity. 
Response  error  persists,  then,  as  the  single  largest  element  capable  of  injecting 
bias  into  the  survey.  Using  a  single  interviewer  and  a  written  interview 
guide/response  sheet  safeguarded  consistent  interview  format  and  accurate  data 
collection. 

The  interview  guide/response  sheet  also  included  the  six  determinant 
attributes  used  by  Brooks  in  his  1979  modeling  effort.  Each  participant  rated  the 
criteria’s  utility  on  a  line  mark  scale  labeled  at  the  extremes  with  low  utility  or 
high  utility.  Ultimately,  the  respondents’  opinions  on  lateral  airfoil  technology 
provide  insight  into  the  true  determinant  attributes  used  in  the  1993  Wright 
Laboratory  decision  making  process.  Once  all  criteria  are  known,  then  the  most 
frequently  used  or  more  important  factors  can  be  distilled.  The  SAS/STAT  package 
provides  descriptive  and  variabUity  statistics  for  this  data  investigation.  The 
frequency  rating  given,  when  compared  with  other  determinant  attribute’s  frequen¬ 
cy  ratings,  shows  the  criteria’s  utility.  In  the  case  of  a  tie  between  two  or  more 
criteria,  the  mean  rating  serves  as  the  discriminate. 

3.5  Question  4:  Effect  of  Cost  Estimate 

What  is  the  part-worth  utility  of  an  engineering  development  cost  estimate  in 
the  decision  maker^s  choice  on  a  lateral  airfoil  design? 


As  noted,  question  three’s  interview  guide  includes  the  six  determinant 
attributes  used  in  Brooks’  Wright  Laboratory  study.  Additionally,  it  included  a 
seventh  potential  determinant  attribute,  a  cost  estimates  for  the  lateral  airfoil 


aeroepaoe  vehicle  of  $210  million.  Each  of  the  thirty-six  subjects  rated  their 
opinion  of  the  engineering  development  cost  estimate  on  the  same  type  of  one 
hundred-fifty-millimeter  line  mark  scale  described  earlier. 

SAS/STAT  procediires  calculated  the  mean  point  estimate  utility  score  for 
the  engineering  development  cost  attribute.  Comparing  the  a^t  estimate’s  mean 
utility  score  with  the  other  six  determinant  attributes  rated  at  the  same  time,  by 
the  same  subjects,  should  reveal  its  utility.  A  higher  point  estimate  of  the  mean 
utility  score  than  Stahl  or  Harrell’s  original  six  criterion  suggests  the  decision 
maker  feels  that  the  criterion  has  greater  utility.  likewise,  a  lower  mean  score 
than  other  determinant  attributes’  shows  lower  utility  in  the  opinions  of  the 
participants. 

3.6  Chanter  Stitnmarv 

This  chapter  detailed  the  methodology  used  to  explore  the  sjjecific  issue  and 
answer  the  research  questions.  Overall,  the  research  methodology  synthesizes  the 
answers  through  a  battery  of  techniques  including  patent  search  bibliometrics, 
mailed  surveys,  and  personal  interviews.  The  next  chapter  displays  the  results  of 
the  surveys  and  personal  interviews  along  with  an  analysis  and  discussion. 
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IV.  Results  and  Discussions 


4.1  Introduction 

This  chapter  offers  a  discussion  and  analysis  for  both  the  survey  and 
interview  responses.  The  siurvey  instrument  described  in  Chapter  III  and  included 
in  Appendix  D  were  mailed  June  4, 1993.  Mass  mailing  followed  coordination  with 
the  Wright  Laboratory  Commander  and  the  chief  scientist.  Appendix  D  also 
contains  a  copy  of  the  cover  letter  that  introduced  both  the  stirvey  and  interviews. 
The  interviews,  following  closely  on  the  heels  of  the  survey,  were  conducted 
throughout  the  weeks  of  Jime  7, 1993  through  Jime  28, 1993. 

Three  hundred  thirteen  sturveys  were  mailed  to  Wri^t  Laboratory  person* 
nel  working  at  Wright-Patterson  Air  Force  Base,  Ohio  and  Eglin  Air  Force  Base, 
Florida.  One  hundred  thirty-two  usable  surveys  were  returned  within  thirty  days 
of  the  mailing.  The  survey  return  rate  of  42.2  percent  is  consistent  with  other 
written  survey,  mass  mailing  return  rates.  While  most  respondents  completed  all 
three  sections  in  the  instruments,  a  few  sxuveys  were  returned  with  one  or  more 
sections  incomplete.  Completion  of  at  least  one  survey  section  served  as  a  mini¬ 
mum  requirement  for  its  measurements’  inclusion  in  the  analysis  data  base.  Some 
surveys  were  returned  with  either  blank  pages  or  an  explanatory  note.  The  notes, 
if  any,  were  excluded  from  the  response  data  base  and  the  survey  was  eliminated 
firom  the  count  of  returned  documents.  This  chapter  also  includes  an  analysis  and 
discussion  of  data  exchanged  during  the  personal  interviews.  Demarcation  and 
utility  measurement  of  decision  making  criterion  for  laboratory  personnel  at 
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Wright  Patterson  AFB,  Ohio  served  as  the  focus  for  the  thirty-six  interviews.  The 
survey  and  interview  results  presented  in  tables,  graphs,  and  narrative  format 
throughout  this  chapter  provide  a  basis  for  summarizing  the  decision  making 
criteria  and  formal  methods  used  in  a  large  United  States  government  research 
and  development  laboratory. 

4^  Demographic  Data 

Demographics  presented  throughout  the  adjoining  pages  define  range 
boundaries  for  survey  respondents  only.  Care  should  be  taken  not  to  confuse  the 
survey  demographics  with  interview  demographics.  In  fact,  few  demographics  were 
collected  diiring  the  exploratory  interviews  since  the  endeavor  focused  on  Wri^t 
Laboratory  as  a  whole.  The  siirvey  respondents  provided  demographic  data 
including  their  age,  civilian  grade  or  military  rank,  gender,  educational  level,  and 
organizational  affiliation.  This  data  appears  in  tables,  graphs,  and  narrative 
format  throughout  the  next  several  pages.  Please  note  that,  where  applicable, 
appropriate  comparisons  with  the  earlier  research  results  of  Prince  (1985)  and 
Congdon  (1988)  appear  with  the  1993  results.  These  comparisons  establish  trends 
for  Wright  Laboratory  from  1985  through  1993. 

Figure  8  and  Table  1  detail  respondent  distributions,  separating  them  by 
level  within  Wright  Laboratory.  A  full  40  percent  of  the  1993  respondents  worked 
at  the  middle  organizational  tier  or  branch  level.  Identical  numbers  of  responses 
returned  fiom  both  the  upper,  division  level  and  lower,  section  levels. 

Earlier  researchers  studying  Wright  Laboratory  used  differing  methodo¬ 
logies.  Prince  chose  a  sample  of  ten  people  each  from  the  Avionics,  Aero  Propul¬ 
sion  &  Power,  Flight  Dynamics,  and  Materials  Directorates.  His  sole  criterion  for 
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a  partidpant  was  "that 
the  individual  be  in¬ 
volved  in  selection  of  an 
in-house  research  pro¬ 
ject"  (Prince, 

1985:23).  Congdon 
mailed  his  survey  to  a 
"non-random  sample  of 
respondents  occupying 
positions  of  Deputy  Divi¬ 
sion  Chief  or  higher" 

(Congdon,  1988:35).  While  the  1988  siirvey  instrument  compares  favorably  with 
the  1993  version,  the  sample  mailing  popvdation  differs.  The  1993  survey  mailing 


Figure  8.  Surv^  Respondents  by  Organisational  Level 


54 


list  included  a  ten-fold  increase  in  sample  size  for  a  broader  representative  cross 
section  of  today's  larger  Wright  Labora^ry  organization.  It  included  section, 
branch,  division,  and  directorate  levels.  The  1993  cosmopolitan  sample  provides 
greater  confidence  that  survey  results  generalize  to  Wright  Laboratory  as  a  whole. 
Unfortunately,  this  expanded  generalization  ability  sacrifices  some  direct  response 
comparisons  with  the  earlier  research.  However,  the  1993  results  should  compare 
favorably  with  other  large  corporate,  educational,  and  Department  of  Defense 
research  organizations. 

4Aa  Agft  Data.  The  distribution  of  survey  respondents  by  age  shows 

nearly  60  percent  of  the  personnel  range  in  age  from  46  through  age  55  and  over. 

A  small  firaction  of  the  respondents  came  firom  the  age  groupings  20  •  30.  This 
ratio  shows  that  the 
respondent  share  a  pro¬ 
pensity  for  a  seasoned 
perspective  on  R&D  tech¬ 
niques.  As  the  1993 
results  are  compared 
with  prior  years,  please 
recognize  that  subtle 
population  differences 
may  obscure  or  exacer¬ 
bate  genuine  variation  in 

respondent  distributions.  Figure  9.  Survey  Respondents  by  Age 
For  instance,  Congdon 


recorded  no  respondents  below  the  age  of  thirty-five  and  only  one  response  through 
age  forty.  The  1993  survey  results  show  20  percent  of  the  respondents  in  these 
categories.  Clearly,  Congdon’s  age  distribution  results  firom  the  senior  supervisors 
sampled,  the  Deputy  Division  Chie&  or  higher,  who  are  usually  older.  These  age 
distributions  influence  interpretation  and  comparison  of  formal  technique  famil¬ 
iarity  and  usage  responses  presented  later  in  this  chapter. 

4»2«b  Gender.  Do  all  respondent  aberrations  stem  firom  elementary 

sample  population  age  differences?  No,  consider  the  respondent’s  gender  as  an 
example.  The  1993  survey  response  distribution  includes  a  7.75  percent  female 

population  (see  Figure  10).  Congdon 
secured  zero  female  responses  with  his 
survey  (Congdon,  1988:46).  It’s  im- 
known  if  this  difference  produces  any 
measurable  impact  since  8  percent  is 
such  a  small  population  change.  How¬ 
ever,  subtle  sample  differences  like 
gender  generate  repercussions  firustrat- 
ing  direct  comparisons  between  this 
and  prior  research  and  necessitate  a 
certain  degree  of  correlation  error  be¬ 
tween  1985, 1988,  and  1993. 

Other  researchers  may  also  have  an  interest  in  this  gender  proportion 
which  further  explains  its  inclusion  here. 


Male 


F^ure  10.  Respondents  by  Gender 
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4J2.C  Directorate  Representation.  Staff  organizations  present 

another  arena  for  dissimilarity  throuidi  the  years  of  data  collection  on  Wright 
Laboratory.  Figure  11  aod  Table  2  show  the  distribution  by  organization  as  a 


Survey  Respondents  by  Directorate 


Man  Tech  (7.7%) 
Armamant  (1 7  0%) 


SoGd  State  (7.0%) 

Materials  (13.4%) 


Aero  PropJsion  (1Q8%) 

•Avionics  (11.6%) 

Financial  Mgit  (3.9%) 

R&D  Ccrtracting  (23%) 
Plans  &  Programs  (4.6%) 
Oth«r  (1 .5%) 

Fight  Dynanics  (20.1  %) 


Figure  11.  Survey  Respondents  by  Directorate 


percent  of  total  respondent  population.  One  can  see  that  Flight  Dynamics  Direc¬ 
torate  personnel  account  for  20  percent  of  the  respondents. 

The  1993  survey  results  appearing  in  this  figure  include  the  staff  oi^aniza- 
tions.  This  differs  &om  prior  researchers.  Prince  solicited  an  equal  ratio  of  respon- 
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dents  firom  Avionics,  Aero  Propulsion  &  Power,  Flight  Dynamics,  and  Materials 
Directorate.  Congdon  included  these  and  other  organizations,  but  excluded  staff 
structures  firom  his  survey  (Congdon,  1988:36).  Typically,  Congdon’s  missing  staff 


Distribution  of  Respondents  by  Organization 


Organization 

Frequency 

Percent  | 

Aero  Fropuision 

14 

10.94  1 

Avionics 

15 

11.72 

Flight  Dynamics 

26 

20.31 

Materials 

16 

13.60 

Solid  State  Electronics 

9 

7.03 

Armament 

22 

17.19 

Manufacturing  Tech 

10 

7.81 

Financial  Management 

5 

3.91 

R&D  Contracting 

3 

2.34 

Plans  &  Prc^rams 

6 

4.69 

Other 

2 

1.56 

Total 

128 

100.00 

Table  2.  Distribution  of  Respondents  by  Directorate 

organizations  encompass  R&D  Contracting  and  the  Operations  and  Support 
Directorates.  However,  please  note  that  the  1993  respondent  distributions  firom 
the  staff  accoxint  for  only  3.9  percent  of  the  total  respondent  poptilation. 
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4^.d  Degree.  This  statistic  reveals  a  very  educated  sample  population 

for  1993  (see  Figure  12).  The  respondent’s  distribution  by  highest  educational  de¬ 
gree  shows  that  80  per¬ 
cent  possess  a  master's 
d^ree  or  higher  educa¬ 
tional  level.  This  demo¬ 
graphic  reveals  a  great 
deviation  from  the  labo¬ 
ratory’s  published  real 
statistics  of  48  percent 
(see  Figure  3).  It  sug¬ 
gests  that  any  decision 
making  technique  famil¬ 
iarity  and  uss^e  percent¬ 
ages  presented  later  may 
disproportionately  reflect 
the  opinions  of  Wright  Laborator3r’s  more  educated  personneL  One  would  assxune 
that  such  a  well  educated  cross  section  would  possess  an  extensive  exposure  to  a 
variety  of  decision-making  methods.  However,  this  assumption  proves  invalid 
considering  data  presented  later  in  this  chapter.  Figure  13  suggests  a  steady  de¬ 
crease  in  Wri^t  Laboratory  senior  educational  levels  since  1985.  However,  the  de¬ 
creased  ratio  of  Doctorate  degrees  may  result  from  1993’s  inclusion  of  a  lai^r 
number  of  personnel  below  the  Deputy  Division  Chief  level. 


Survey  Respondents  by  Degree 
Barger  (199:^ 


Figure  12.  Survey  Respondents  by  Degree 
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Figure  13.  Summary  of  Respondents  by  Highest  Educational  Degree 

4«2»e«  Dc|?rce  Major,  Some  mii^t  question  whether  the  degree 

msgor  represents  an  important  demographic  for  discussion.  If  one  contends  that 
certain  Masters  level  educational  curricula  should  include  courses  in  Laboratory 
Management,  Operational  Science,  or  Dedsion-making  Methods,  then  these 
statistics  may  prove  useful. 

Most  of  today’s  Wright  Laboratory  respondents  majored  in  an  engineering 
discipline  (see  Figure  14  and  Table  3).  This  tracks  well  with  the  laboratory’s 
published  demographics  showing  an  80  percent  engineering  degree  research  force 
(see  Figure  2).  Comparing  this  study  with  prior  research  reveals  that  it  tracks 
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Survey  Respondents  by  Degree  Major 


Materials/Metaflurgy  (&6%) 

Management  (4.8%] 
Chemistry  (7.2%) 
Psychology  (2.4%) 

{Business  Related  (11.2%) 

Math  (a8%) 

Other  (5.6%) 

Physics  (4.0%) 


Engineering  (5a  4%) 


Figure  14.  Survey  Respondents  by  Dupree  Magor 


closer  with  today’s  laboratory  population  than  Piinoe  who  idratified  65  percent 
his  respondents  as  engineers  (Prince,  1985:48).  The  second  most  fashionable  1993 
inajor  was  a  business-related  study.  Please  note  that  despite  its  number  two 
ranking,  business  and  management  msgors  accounted  for  only  16  percent  of  the 
total  respondent  population. 
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Oistribatiua  of  Respondents 
Mayor  Field  of  Study  in  Hipest  Degree  Granted 


Table  3.  Distribution  of  Survey  Respondents  by  Degree  Mayor 
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4«2Jr  Coiir8C"WOrk«  LaboratOTy  management  cuixicula’s  mwiplfttimi 
rate  was  at  nearly  80  percent  for  1993  (see  Table  4).  This  high  figure  for  such  a 


Distrilmtion  of  Pespcmdents  by  ^>plicable  Course-woik 

1 

■ 

Courses 

Frequency 

Percent 

Prequoicy 

Percent 

1 

Attended 

"Yerf* 

-Ye^ 

-No" 

-No" 

1 

MStOR 

79 

61.24 

50 

38.76 

1 

Tjih  Management 

100 

77.52 

29 

22.48 

1 

Table  4.  Distrilmtum  of  Survey  Beqpondents  by  Applicable  Courae-woik 


widely  distributed  sample  displa3rs  a  sound  educational  background.  Differences 
among  this  study  and  Prince  and  Congdon’s  research  efforts  could  be  sample  mze 
d^ndent  (see  Figure  15).  If 
the  decrease  varies  with  the 
sample  population,  then  other 
related  indices  should  exhibit  a 
comparable  10  percent  decline. 

Interestingly,  the  percentage  of 
course  work  completed  in  Man¬ 
agement  Sdence/Operational 
Research  has  fallen  by  20  per¬ 
cent  in  the  same  period  compar¬ 
ing  the  same  sample  popula¬ 
tions  (see  Figure  16). 


Course  Work  Completed 

Laboratory  Management 


^  yes  r~~1  no  j 


Figure  15.  Percentage  Respondents  Completed 
Laboratory  Management 
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Potratial  explanatUms  for  this  inconsistency  span  a  range  from  the  pessimistic 
tenet  suggesting  the  results  spring  from  a  general  disenchantment  with  manage¬ 
ment  sdence,  operations  research  and  laboratory  management  ttianipKnfta  to  the 
positive  hope  that  the  res\ilts  reflect  a  growing  d^p«e  of  qiedalizatian  within 
these  fields.  Whatever  the  underlying  cause,  these  demographics  point  to  a 
potential  for  decreased  familiarity  with  formal  R&D  project  selection  techniques 
simply  because  people  have  had  fewer  exposure  opportunities  for  the  materials. 
Interestingly,  the  next  section  shows  that  an  mspiring  percentage  of  respondents 
said  they  would  be  "willing  to  attend  a  seminar  on  formal  R&D  project  selection 
techniques." 


Percentage  Respondents  Completed 

Management  Science/Operational  Research 


Prince  <198^  Congdon  (1988)  Barger  (1993) 
^  y«5  ^  no 


Figure  16.  Percenti^  Respondents  Completed  MS/OR 
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4^.g  Wi 


Uingness  to  Attend  Course,  In  the  1993  survey,  76 

percent  of  the  Wright  Laboratory  population  responded  positively  to  this  query 
(see  Figure  17).  This  statistic  suggests  a  keen  interest  in  the  topic  and  compares 
positively  with  80  percent  identified  by  Congdon  in  1988.  One  rational  for  toda/s 
interest  may  stem  from  the 
large  number  of  respondents,  91 
percent,  who  acknowledged  that 
a  technique  might  prove  benefi* 
dal  to  them.  Comparison  with 
other  researchers  suffers  some¬ 
what  on  this  demc^raphic  since 
Pxinoe’s  statistics  were  void  of 
information  on  respondent  will¬ 
ingness  to  attend  R&D  Project 
Selection  classes. 


Pefcantaga  Respondents  WiKng  to 
Attend  a  Protect  Selection  Course 


iootT 


O 

1 

a 

% 

0. 


Congdon  (I98e 


Barger  (1903) 


YES  EH  NO 


Figure  17.  Percentage  Bespondents  ^^lling  to 
Attend  a  Course 


4^ Jl  GritdA  nr  Ravik.  Please  recall  that  both  the  1988  and  1993 

survey  instruments  were  essentially  identicaL  However,  some  data  from  1988  is 
unavailable  for  oompaiison.  One  such  measurement  missing  from  Congdon’s 
research  is  respondent  rank  or  dvilian  grade.  Since  these  demographics  may  prove 
useful,  the  next  few  pages  present  it  and  other  important  indicators  such  as  budget 
responsibility  and  discretionary  research  percentages.  These  demographics  have  a 
potential  impact  on  survey  comparisons  with  Prince  and  Congdon’s  research. 
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Wright  Laboratory  serves  the  Uzuted  States  Air  Force  as  one  <^its’  four 
owned  and  operated  "super  laboratories.”  One  iinfamiKar  with  these  national 


Figure  18.  Bespondent  Percentage  by  Grade  or  Rank 


resources  might  assiune  that  their  military  posture  dictates  a  purely  military 
population.  In  reality,  most  of  Wright  Laboratory's  proletarian  remain  civilian. 
The  published  ratio  of  civilian  to  military  for  the  laboratory  gives  an  84  percent 
civilian  population  (Wright  Laboratory,  1992).  The  1993  survey  respondents 
match  this  population  figure  giving  an  85  percent  civilian  population  (see  Table  5 
and  Figure  18).  Prince,  despite  sampling  a  difTerent  population,  shares  this  obser¬ 
vation,  recording  nearly  an  equal  percentage  of  personnel  (87.5%)  in  1985 
(Prince,  1985:46). 
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Distribution  of  Respondents  by  Civilian  Grade  or  Military  Rank 


Tsblc  5.  Distribution  of  Survey  Respondents  by  Military  Grade  or  Civilian 
Rank 
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4^»i  Budget.  As  shown  in  Figure  24,  85  percent  of  the  respondents  re¬ 
plied  that  they  were  responsible  for  at  least  $1  million  annual  budget  authority. 
This  contrasts  with  Prince  who  found  60  percent  responsible  for  $1  million  or  more. 
Hbwevmr,  Prince  uses  1985  dollars  while  this  study  uses  1993  as  the  base  year. 

The  large  sums  of  money  managed  reflects  the  heavy  educational  level  and 
research  expertise  required  and  also  impacts  both  civilian  and  military  grade 
structures  within  Wright  Laboratory.  Readers  familiar  with  grade  structures  may 
have  noticed  an  absence  of  military  enlisted  rank,  Wage  Grade  or  lower  tier  GS 
personnel  in  the  data  presented. 
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4»2«1  Ill«HottS6  Research.  One  Half  the  respondents  indicated  that 

30  percent  or  more  of  their  budgets  were  spent  in-house,  while  13  percent  contract¬ 
ed  out  90  percent  or  more  of  their  research.  Figure  20  shows  that  a  majority  of 
respondents  reported  a  large  contract  budget  which  compares  favorably  with  a 
published  laboratory  statistics  of  76  percent  (Wright  Laboratory  Fact  Sheet,  1992). 


1CD  90  80  70  60  50  40  30  20  10  N/A 

Percentage  on  Contract 


Figure  20.  Respondent  Percentage  by  In-house  Research 
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Discretionary  Research  One  half  the  respondents 


indicated  that  less  than  70  percent  of  their  division’s  research  was  discretionary, 
or  was  not  specifically  requested  by  a  SPO  or  outside  organization  (see  Figure  21). 
Whatever  the  reason,  this  response  distribution  suggests  that  a  formal  R&D 
project  selection  tool  might  prove  useful  70  percent  of  the  time  for  at  least  half  the 
people  surveyed.  This  information  will  carry  important  implications  as  the 
research  questions  are  explored  and  answered  throughout  the  remainder  of  this 
chapter. 


10  20  30  40  50  60  70  80  90  100  N/A 

Percentage  of  Projects  Requested 


Figure  21.  Percentage  of  Discretionary  Research 
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4J2Jl  Technique  Awareness.  Table  6  and  Figure  22  summarize 

the  tally  from  survey  questions  one  through  sixteen.  These  series  of  questions 
measured  respondent  familiarity  with  some  accepted,  formal  R&D  project  selection 
techniques.  It’s  readily  apparent  that  over  50  percent  of  today’s  respondents  profe¬ 
ssed  literacy  with  at  least  three  methods.  Checklist,  Cost/Benefit  Ratio,  and 


Survey  Respondent  Percentages  for 
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Figure  22.  Respondent  Percentages  for  Technique  Awareness 


Decision  Tree  techniques  make  up  today’s  top  three  best-known  methods  for 
Wright  Laboratory.  Goal  Programming,  Integer  Programming,  and  Portfolio 
Models  aU  competed  for  the  dubious  distinction  of  least-well-known”  technique. 
One  might  presume  a  stronger  prowess  for  Portfolio  Models  in  a  big  laboratory 
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Familiarity  with  and  Use  of  R&O  Project  Selection  Techniques 


Technique 

Not  Familiar 

Familiar 

Regular  Use 

(Percent) 

(Percent) 

(Percent) 

Q-Sort 

57.81 

42.19 

14.84 

Checklist 

41.41 

58.59 

24.22 

Scoring  Models 

54.69 

45.31 

19.53 

Delphi 

57.03 

42.97 

7.31 

Cost/Benefit  Ratio 

39.06 

60.94 

13.28 

Payback  Period 

53.13 

46.87 

. 

7.81 

Net  Present  Value  /IRR 

67.97 

32.03 

2.34 

Portfolio  Models 

89.84 

10.16 

1.56 

Risk  Analysis/Monte  Carlo 

56.25 

43.75 

5.47 

Decision  Trees 

44.53 

55.47 

10.94 

Linear  Programming 

64.84 

35.16 

3.91 

Integer  Programming 

88.28 

11.72 

.78 

Nonlinear  Programming 

82.03 

17.97 

.78 

Dynamic  Programming 

82.81 

17.19 

.78 

Goal  Programming 

86.72 

13.28 

3.13 

Table  6.  Technique  Distributions  for  Awareness  and  Use 
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environment,  yet  both  the  1993  and  1988  resiilts  show  equally  low  ratings  for  this 
method. 

The  top  three  techniques  for  1993  reflect  change  firom  1988  (see  Figure  23). 
Please  note  that  familiarity  with  the  more  "user  friendly"  techniques,  Sort  and 
Checklist  has  grown  at  the  expense  of  more  rigorous  methods  like  Risk  Analysis, 
Linear  Programming,  Decision  Tree  and  I^lphi  Methods.  Prince  noted  this  same 
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Figure  23.  Technique  Familiarity  for  1988  and  1993 


phenomena  writing  that  the  1988  respondents  used  "relatively  simple  to  learn  and 
easy  to  use  techniques"  (Prince,  1988:38).  Interestingly,  respondent  familiarity 
with  Net  Present  Value  and  Internal  Rate  of  Return  has  doubled  since  1988  while 
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Cost/Benefit  Ratio  has  dropped.  Still  nearly  twice  as  many  people  remain  familiar 
with  Cost/Benefit  Ratio  today. 

Most  Wri^dit  Laboratory  respondents  describe  Cost/Benefit  Ratio  as  their 
best  known,  technique.  Does  technique  literacy,  then,  serve  as  a  good  predictor  of  a 
procedure’s  useRilness  in  a  r.-^al  decision  making  environment?  Will  the  survey 
results  show  an  overwhelming  usage  rate  for  the  Cost/Benefit  technique?  Subse¬ 
quent  sections  address  this  issue,  prompting  a  surprising  finding.  The  special 
emphasis  on  Cost/Benefit  Ratio  in  this  section,  then,  lays  the  groundwodk  for  an 


important  observation  later  on  the  subject  of  familiarity  as  a  predictor  value. 


Figure  24.  Comparison  of  Awareness  for  1985, 1988, 1993 
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Congdon  expressed  concern  at  the  low  level  of  formal  method  familiarity 
considering  the  ratio  of  respondents  completing  Master's  degrees  (Congdon, 
1988:62).  Today’s  respondents  exhibit  higher  educational  levels  with  still  lower 
total  familiarity  percentages.  If  C^ongdon’s  comments  suggest  an  educational 
curriculum  deficiency,  then  the  educational  system  has  facilitated  little  improve¬ 
ment  in  this  statistic  diuring  the  last  one-half  decade. 

Figure  24  displays  respondent  familiarity  on  shared  techniques  surveyed  in 
1985, 1988  and  1993.  The  graph  highlight’s  fluctuation  through  the  years  with 
many  well  documented  methods,  like  Goal  Programming. 

4.3  Technique  Use 

Why  were  so  many  demographics  a}llected  and  described  in  the  preceding 
pages?  Constructing  a  true  picture  of  the  laboratory  hinges  upon  an  accurate  mea¬ 
surement  of  the  distortions  inherent  in  the  research  results  collected  through  the 
years,  using  the  differing  methodologies  and  different  siurvey  samples.  To  the 
extent  possible,  differences  in  techniques  springing  firom  methodology  or  sample 
differences  are  identified. 

Pie  charts  in  Figures  25,  26,  and  27,  show  formal  methods  usage,  as  a  per¬ 
centage  of  respondents,  for  three  study  years  1985,  1988,  and  1993.  Prince  found 
30  percent  of  the  population  employed  a  formal  method  in  1985.  In  1988,  only  one 
in  foiu:  of  the  personnel  surveyed  used  a  formal  technique.  Remembering  the 
sample  population  differences  between  Prince  and  Congdon,  one  conceivable 
explanation  for  the  statistic  stems  fix>m  the  larger  numbers  of  lower  organizational 
participants  used  in  1988.  However,  maintaining  this  logic  suggests  that  the  1993 
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Technique  Use  in  1985 
(Prince) 


Technique  Use  in  1988 
(Congdon) 
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results  should  be  lower  still.  Interestingly,  nearly  50  percent  of  the  1993  respon¬ 
dents,  when  asked  if  they  used  a  technique,  responded  positively.  The  1993  resiilts 
show  neaziy  a  two-fold  growth  in  just  five  years.  This  statistic  ofiers  a  surprise 
since  today’s  respondents  typically  include  workers  at  lower  organizational  levels. 


Technique  Use  in  1993 

(Barger) 


Figure  27.  Technique  Use  in  1993 

Further,  these  same  respondents  have  expressed  lower  total  familiarity  with  the 
various  dedsion-makuig  methods  available. 

The  distribution  of  respondents  using  formal  methods  when  grouped  by 
organizational  level  manifests  an  almost  linear  relationship  between  percentage 
use  and  hierarchical  position  (see  Figure  28).  The  "directorate  level”  personnel 
exhibit  twice  the  likelihood  of  using  a  formal  method  as  Wright  Laboratory’s  "sec¬ 
tion  level"  personneL  Indeed,  many  would  argue  that  persoimel  at  the  section  level 
should  focus  on  research  and  not  decision-making.  Unfortunately,  this  view  does 
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not  consider  a  &nnal  techniques’  applicability  to  an  individual’s  routine  decision¬ 
making  process. 

In  clear  contrast  to  the  linear  relationships  displayed  across  lahoratory 
hierarday,  there  exists  substantial  diversity  in  usage  percentages  between  director¬ 
ates  (see  Figure  29).  As  one  might  expect,  the  Plans  &  Programs  Directorate 


Technique  Usage  by  Organizational  Level 
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Figure  28.  Technique  Usage  by  Oiganizational  Level 
exhibits  a  hig^x  usage  propensity.  The  directorate’s  small  size  contrasts  with  its 


enormous  task,  plotting  the  laboratory’s  course  throu^  the  20th  century’s  high 
technology  fix>ntier.  However,  the  Plans  &  Programs  Directorate’s  task  lends  itself 
well  to  formal  technique  usage.  Within  the  Wright  Laboratory  directorates  as  a 
whole,  great  gul&  abide  in  technique  application.  For  example.  Materials  Directo- 
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rate’s  formal  technique  usage  far  outstrips  either  Armament  or  Solid  State 
Electronics  Directorate’s  application  levels. 

A  statistical  procedure  was  used  to  test  for  a  diversity  in  usage  phenomena. 
The  small  sample  size  of  six  respondents  in  one  directorate  coupled  with  an 
unknown  potential  for  normal  data  distribution  overall  made  the  Kruskal- Wallis 
test  appropriate.  Acceptance  of  the  null  hypothesis  occurred  if  usage  levels  were 
statistically  identical  at  an  alpha  level  of  0.10.  This  test  supported  the  alternate 
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'igure  29.  Percentage  Using  Technique  by  Directorate 


hypothesis  that  usage  levels  across  directorates  were  not  the  same.  An  value  of 
12.4  exceeded  the  test  statistic  of  12.02,  at  seven  degrees  of  freedom,  with  a  p- 
value  of  .09.  One  concludes,  then,  that  at  least  two  of  the  eight  directorates  tested. 
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wielded  formal  techniques  differently.  These  statistical  results  appear  reasonably 
concrete,  but  one  need  only  study  the  following  graphs  to  intuitively  confirm  the 
mathematics. 

Individual  Directorates’  supporting  or  opposing  rationale  for  formal  meth- 
od(s)  use  lay  beyond  this  research  effort’s  domain.  Yet,  the  survey  data  clearly 
show  that,  whatever  the  reasons  for  using  or  not  employing  a  formal  decision 
method,  technique  knowledge  does  not,  necessarily,  propitiate  technique  use.  The 
Solid  State  Electronics  Directorate  serves  as  an  example  in  this  query.  Their 
responses  su^ested  they  were  cc^n^^u^t  of  many  formal  techniques.  Fifty  percent 
of  the  respondents  recorded  fiuniliarity  with  the  cost/benefit  ratio  technique.  In 
fact,  many  were  conversant  with  other  methods  as  well.  Yet,  no  one  used  the 
cost/benefit  method  in  the  directorate.  However,  the  following  caution  is  in  order: 
failure  to  use  a  particular  technique  is  not  the  issue  here.  The  driving  issue  is 
whether  there  is  an  imknown  potentiaKs)  for  any  one,  or  many,  formal  technique(s) 
to  be  inappropriate  in  a  given  application.  It  appears  that  the  cost/benefit  tech¬ 
nique  is  not  an  effective  method  for  Solid  State  Electronics  directorate.  Building  on 
this  thought,  while  realizing  that  Wright  Laboratory  exhibits  strong  knowledge  and 
usage  levels  for  formal  techniques,  one  can  question  if  the  organization  feels  that 
any  or  all  techniques  lend  themselves  to  laboratory  use.  In  the  case  of  the  Solid 
State  Electronics  directorate,  perhaps  the  cost/benefit  technique  does  not  easily 
adapt  itself  to  their  decision  conditions. 

A  Wilcoxon  rank  sum  test  was  used  to  probe  for  disagreement  on  this  issue 
within  Wright  Laboratory.  The  null  hypothesis  that  the  directorates  agreed  in 
their  opinion  on  likelihood  that  a  formal  project  selection  technique  which  would  be 
useful  during  project  selection  and  resource  allocation  could  be  developed,  was 
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supported.  The  Kruskal-Wallis  test  (Chi-Square  Approximation)  was  7.4834  with 
seven  degrees  of  freedom  and  p=.38.  Therefore,  the  directorates  appear  to  be  in 
agreement  on  the  issue.  Interestingly,  the  Kruskal-Wallis  test  confirmed  a 
directorate  agreement  that  the  likelihood  of  a  technique  being  developed  was 
"uncertain."  However,  Solid  State  Electronics  respondents  were  more  pessimistic, 
deeming  the  likelihood  as  closer  to  "unlikely!"  Their  mean  score  was  the  largest  of 
all  compared  on  this  issue. 

These  statistics  suggest  the  directorates  are  uncertain  if  a  formal  method 
can  be  developed  to  meet  all  their  needs.  However,  they  do  employ  some  well- 
known  formal  techniques  as  appropriate.  Further,  as  subsequent  sections  show, 
the  directorates  employ  the  myriad  of  techniques  to  dissimilar  degrees. 

Comparing  the  survey  results  for  technique  usage  by  age  revealed  non-use 
by  the  younger  respondents  between  20-25  (see  Figure  30).  However,  the  low 
number  of  responses,  only  7  percent,  for  population  between  the  age  span  between 
20  and  30  impeirts  low  reliability  to  this  measinre.  After  age  25,  technique  usage 
increases  through  age  40.  Between  ages  40  and  50  application  tapers  off  to  levels 
of  less  than  30  percent,  but  rebounds  to  higher  levels  prior  to  age  55. 

Statistics  on  formal  technique  usage  by  military  rank  reveals  that  Captain 
through  Lieutenant  Colonel  rank’s  advocate  technique  usage  more  than  Colonels 
(see  Figure  31).  Again,  the  low  numbers  of  respondents,  six  people  in  the  Lieuten¬ 
ant  and  Colonel  ranks,  give  this  measure  questionable  merit.  In  the  mid  ranges 
the  data  provides  a  higher  reliability  and  becomes  more  indicative  of  the  poptila- 
tion.  The  civilian  grades  exhibited  less  percentage  of  use  at  the  upper  tier  when 
GS  and  GM  categories  are  combined,  but  again  the  statistical  reliability  suffers 
because  of  few  respondents  at  these  higher  levels  (see  Figure  32). 
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Technique  Usage  by  Age 


Figure  30.  Technique  Usage  by  Age 


» 
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Percentage  usage  within  group 


echnique  Usage  by  Military  Rank 


Figtire  31.  Tecimique  Usage  by  Rank 
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Percentage  Use  by  Civilian  Grade 


Figure  32.  Percentage  Use  by  Civilian  Grade 
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A  ribbon  chart  in  Figure  33  confirms  that  the  probability  of  formal  tech¬ 
nique  usage  in  a  decision  process  strides  to  100  percent  as  the  budget  involved 


approaches  $1  Billion.  However  at  lower  dollar  ranges,  formal  techniques  can 
function  in  roles  other  than  monetary  resolution.  Some  of  these  roles  are  exam¬ 
ined  in  the  next  section.  A  deeper  discussion  of  published  decision  making 
techniques’  strengths,  weaknesses,  and  authoritative  references  appear  in  Appen¬ 
dix  B.  Jordan’s  Naval  Postgraduate  thesis  provides  an  additional  information 
source.  He  wrote  fi:t>m  the  military  managers’  perspective  (Jordan,  1992). 
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What  methods  do  managers  use  to  select  R&D  projects  and  allocate  laboratory 
resources  at  Wright  Laboratory? 

The  previous  section  established  that  technique  usage  varied  between 
directorates.  Additionally,  individiial  directorates  embraced  different 
of  methodologies.  For  example,  the  Plans  &  Programs  Directorate  clearly  preferred 
the  checklist  technique.  Aerospace  Propulsion  &  Power  Directorate,  on  the  other 


Three  Techniques  Used  by  Directorates 

as  a  Percentage  of  Users 


I 

I 
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Plans  &  Programs 
Propulsion 
Mantech 
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Materials 
Flight  Dynamics 
Soiid  State 
Avionics 
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Cost/Benefit  Ratio  Y//A  Decision  Tree 


Figure  34.  Comparison  of  Three  Tedmiqnea  Used  by  Directorates 


hand,  applied  cost/benefit  ratio  two-to-one  over  other  techniques.  Figure  34 
compares  three  formal  methods  used  across  the  directorates.  The  evidence 
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suggests  that  Wright  Laboratory  directorates  exhibit  a  propensity  to  style  unique 
combinations  of  decision-making  methods  to  cope  with  their  distinctive  decision 


climate. 


A  bar  chart  (see  Figure  35),  displaying  some  common  techniques  among 
three  studies,  shows  usage  pattern  fluctuation  among  the  years.  The  metamor¬ 
phosis  of  routines  employed  bint  of  adaptive  decision-making  on  a  macro  scale. 


Figure  35.  Technique  Used  for  1985, 1988, 1993 
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For  example.  Prince  found  great  support  for  goal  programming  in  1985.  By  1988 
the  technique  had  fallen  firom  popularity  while  linear  programming  use  slqnrocket- 
ed.  Then,  in  1993,  linear  programming  use  plummeted  back  to  a  peer  plane  with 
goal  pn^pramming.  Other  charts  displayed  throughout  the  subsequent  pages  (see 
Figures  36-43)  detail  the  unique  mix  of  methods  employed  by  directorate.  Howev¬ 
er,  the  reader  is  cautioned  that  any  comparison  between  1985, 1988,  and  1993 
should  consider  the  differences  in  the  sample  populations  surveyed. 


Figure  36.  Methods  Used  in  Avionics 
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Figure  37.  Methods  Used  in  Aero  Propulsion  &  Power 


Soct(2a8%) 
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Figure  39.  Bifethods  Used  in  Flq^t  Dynamics 


Manufacturing  Technology  Directorate’s 
Techniques 


Ratio  (222%) 


Al  Others  (11.1%) 


Sort  (222%)- 


Figure  40.  Bfethods  Used  in  Bfanofacturing  Tech¬ 
nology 
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Solid  State  Electronics  Directorate’s 
Techniques 


Oth«r(33^‘K) 


tiackllst  <33l3%) 


Score  (33.3^ 


Figure  41.  Methods  Used  in  Solid  State  Electronics 


Figure  42.  Methods  Used  in  Materials 


Plans  &  Programs  Directorate’s 
Techniques 


Figure  43.  Methods  Used  in  Plans  &  Programs 

Throughout  the  preceding  pages  and  sections,  graph  after  graph  descrihes 
the  fluctuating  and  constantly  transforming  decision-making  process  within  Wright 
Laboratory.  The  statistics  also  describe  an  organization  familiar  with  many  formal 
techniques.  Clearly,  Wright  Laboratory  regularly  exercises  numerous  formal  tech¬ 
niques,  while  others,  despite  being  well  tmderstood,  are  discarded.  The  graphs 
reflect  a  large  laboratory  organization  adapting  technique  usage  through  the  years 
and  across  the  organizational  structure  to  meet  its  decision  making  environment. 
Undoubtedly,  Wright  Laboratory  does  not  blindly  enforce  an  ill-fitted  application. 
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nor  does  it  employ  a  technique  simple  because  it  is  well  known  throughout  the 
organization. 


4^  Question  2:  Lateral  Airfoil  ResAar<»li  ValiHItv 

What  establishes  lateral  airfoils  as  a  potentially  viable  research  project^ 

Recall  that  Section  2.5  addressed  this  question  from  a  bibHometric  ap¬ 
proach.  The  narrative  compared  many  patents  granted  for  aerospace  vehicle 

designs  and  de¬ 
scribed  lateral 
airfoil  applications 
for  gas  turbine 
engines  and  naval 
propulsion  sys¬ 
tems.  The  infor¬ 
mation  presented 
within  Chapter  2 
met  the  intent  of 
this  research 
question.  Howev¬ 
er,  in  a  further  test  of  lateral  airfoil  research  potential,  each  of  the  thirty-six 
interview  participants  was  asked  if  they  wotild:  1)  Support  a  lateral  airfoil 
research  project;  2)  Table  the  project  until  some  later  date;  or  3)  Oppose  the 
project  The  decision  makers  marked  their  resjMnse  after  hearing  a  short,  5-7 
minute,  overview  of  the  aerospace  vehicle  design.  The  participants’  information 


Interview  Respondent’s  Support  for 
Lateral  Airfoil  Project 


Oppose  Research  (saX) 


Table  Research  (47.2%) 


Research  (41.7%) 


Other  (28%) 


Figure  44.  Interview  Bespondenfs  Support  for  Lateral  Airfoil 
Project 
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included  an  engineering  development  cost  estimate  of  $210  million.  This  procedxire 
minimized  interview  time  and  placed  each  respondent  at  about  the  same  level  of 
disadvantage  on  lateral  airfoils. 

When  asked,  only  8  percent  of  the  36  participants  truly  opposed  the  project 
(see  Figure  44).  However,  almost  all  participants  verbally  predicated  their  support 
upon  receiving  positive  answers  to  the  decision  criteria  that  they  subsequently 
listed  and  evaluated  on  the  interview  guide.  For  example,  more  than  one  individu¬ 
al  wanted  to  know  if  any  basic  research  had  been  performed  and  if  so,  was  that 
prior  research  successful  enough  to  merit  further  lateral  airfoil  development. 

These  results  support  the  use  of  lateral  airfoil  technology  in  answering  Research 
Questions  3  and  4. 

4.6  Question  3;  Detenmnant  Attributes  Used 

What  determinant  attributes  do  decision  makers  use  in  a  final  choice  set 
during  research  program  portfolio  selection? 

The  reader  may  recall  the  discussion  earlier  of  Doctors  Stahl  and  Harrell 
who  conducted  interviews  with  Wright  Laboratory  personnel  in  the  late  1970’s. 
They  found  six  predicdve  factcms  (determinant  attributes)  commonly  used  in  the 
decision  making  enviroiunent  (see  Figure  45).  They  talked  with  personnel  firom 
the  command  section  through  section  chief  levels,  (generally,  once  the  determinant 
attributes  or  predictor  criteria  are  known,  other  researchers  can  design  experi¬ 
ments  to  model  the  decision  making  process.  Brooks  subsequently  used  Stahl  and 
Harrell’s  determinant  atiributes  to  develop  a  model  of  Wright  Laboratory’s  decision 
making  process  (Brooks,  1979:51).  He  found  that  all  six  factors  identified  by  Stahl 
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Figure  45.  Stahl  and  Harrell's  Determinant  Attributes  GSroofcs,  1979) 

and  Hnrrell  combined  to  form  a  linear  model  accurately  describing  Wright  Labora¬ 
tory  decision  making  process  of  the  late  1970's.  The  six  attributes  ranked  by  order 
of  Brooks’  research  beta  weights  are: 


/.  Air  Force  Need 

2.  Techitieal  Merit 

3.  CostlBeiufit  Ratio 

4.  Resource  AvailoMtity 

5.  LikeShood  of  Success 

6.  Time  Period. 

(Brooks,  /r 


Unfortunately,  no  documentation  exists  explaining  how  Stahl  or  Harrell  identified 
these  criteria.  Therefore,  the  1993  research  process  targeted  this  material  void, 
successfully  identifying  a  greatly  expanded  set  of  criteria.  Additionally  the  same 
criteria  advocated  by  Stahl  and  Harrell  were  reintroduced  to  gauge  their  merit  for 
today. 

The  1993  results  distilled  firom  the  interview  process  where  each  participant 
was  asked  to  list  and  measiue  the  utility  of  their  decision  criteria  in  a  lateral 
airfoil  research  project  decision.  Approximately  sixty  distinctive  criteria  surfaced. 
Some  determinant  attributes  appeared  in  interview  after  interview,  while  other 
criteria  emerged  only  once.  Therefore,  how  can  one  develop  a  fairly  comprehensive 
list  of  possible  decision  attributes  when  such  a  large  number  of  criteria  surfoce? 
One  author  suggests  a  popular  vote  technique,  eliminating  determinant  attributes 
mentioned  by  less  than  5  percent  of  the  participants  (Louviere,  1988:51).  Using 
this  procedure  successfully  reduced  the  number  of  criteria  by  nearly  one-half.  The 
reduced  set  of  1993  determinant  attributes,  the  participants  mean  rating  of  their 
utility,  and  the  niunber  of  times  it  was  identified  appear  in  Table  7. 
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1993  Determinant  Attributes 


Attribute 

Mean 

literature  Review 

123.5 

Alternative  Technology 

116.0 

Expert  Opinion 

1U.2 

Technical  Merit 

112.8 

Within  Discipline 

115.0 

Cost/Benefit 

118.8 

Efficiency  (Performanoe) 

108.8 

Zero  Sum  (against  other  prqj) 

103.3 

Cost  Factors 

114.2  ■ 

Funding  Availability 

97.3 

Risk 

116.8 

Environment 

94.3 

Prior  Supporting  Research 

122.7 

DoD  Need 

115.3 

User  Need 

132.7 

USAF  Mission 

131.8 

Can  it  be  Produced 

96.7- 

Sunk  Costs 

94.0 

Test  Plan 

82.7 

PayofPMission  Potential 

135.0 

Multi-Service  Option 

107.3 

Industrial  Base  Impact 

100.0 

Organic  Facilities 

87.0 

Time  (Schedule) 

103.3 

Meet  10  ILS  Elements 

86.4 

life  Cycle  Costs 

100.3 

Cost  Figure  Accuracy 

98.7 

Technology  Feasibility 

124.3 

Manpower 

94.2 

State  of  the  Art 

130.3 

Table  7.  1993  Determinant  Attributes 
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Trying  to  build  a  decision  model  on  twenty-nine  criteria  may  prove  too 
complicated  for  experimental  testing.  However,  several  techniques  exist  for 
reducing  the  list  further.  For  example,  Louviere  notes  that  participants  may 
propose  composite  or  "fuzzy"  concepts  instead  of  actionable  decision  attributes.  He 
identified  qiiality  and  convenience  as  concepts  fitting  this  category,  suggesting  that 
they  may  carry  different  meanings  to  different  people.  Louviere  advised  using 
conjoint  techniques  to  define  them  in  a  function  (Louviere,  1988:52). 

A  second  option  would  be  to  perform  a  factor  analysis  together  with  a 
varimax  routine,  combining  the  factors  and  the  data  base  values,  and  perform  an 
analysis  of  variance.  One  could  repeat  the  process  until  the  original  listing  of 
criteria  distilled  into  an  orthc^onal  grouping  small  enough  to  test  experimentally. 
Unfortunately,  the  thirty-six  participants  in  the  1993  interviews  provide  a  data 
base  too  small  for  a  quality  analysis  using  factor  analysis. 

A  third  procedure  for  grouping  the  criteria  might  be  correlation  analysis.  If 
a  strong  correlation  existed  between  two  criteria,  then  perhaps  one  could  cancel  or 
completely  explain  the  other,  thus  permitting  elimination  of  one  or  more  determi¬ 
nant  attributes  firom  the  table.  For  example,  a  strong  inverse  correlation,  -0.93693, 
with  a  p-value  of  .0019  surfaced  between  the  determinant  attributes  "risk"  and 
"expert  opinion."  This  statistic  suggests  that  the  decision  makers  felt  a  substan¬ 
tially  reduced  project  risk  in  the  light  of  favorable  expert  opinion.  Therefore,  when 
reducing  a  massive  criterion  listing  such  as  this,  one  need  only  consider  using 
either  "risk"  or  "expert  opinion,"  but  not  both.  Unfortunately,  not  enough  correla¬ 
tions  presented  themselves  to  substantially  reduce  the  1993  list  of  candidate 
criteria  into  a  small  experimentally  acceptable  set 
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Alternatively,  one  could  continue  using  Louviere’s  method  by  raising  the 
cut-off  percentage  threshold  far  above  the  original  5  percent.  Applying  this  routine 
reduced  the  1993  determinant  attribute  list  to  seven  potential  criteria.  The  top 


Figfure  46.  Top  Seven  1993  Determinant  Attributes 


seven  1993  determinant  criteria,  their  mean  rating  of  utility  on  a  scale  form  zero  to 
150,  and  the  number  of  people  voicing  them  are  shown  in  the  Figure  46. 

Mention  should  be  made  of  the  potential  for  variability  in  criterion  usage 
and  beta  wei^t  in  a  large  laboratory  environment.  Brooks  found  that  managers  of 
6.2  level  projects  used  different  criteria  beta  weights  than  managers  of  6.3  prqjects 
(Brooks,  1979:72).  Additionally,  he  determined  that  each  directorate  placed 
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different  relative  weights  upon  determinant  attributes  by  using  a  Chow^s  F-test 
(Brooks,  1979:76).  Further,  Brooks  concluded  that  the  individuals  he  surveyed 
could  not  accurately  specify  the  weights  they  placed  upon  their  attributes 
(Brooks,  1979:81). 

Does  this  mean  that  any  effort  to  isolate  and  measure  determinant  attrib¬ 
utes  is  futile?  No!  However,  any  successful  mnployment  of  Wright  Laboratory’s 
decision  criteria,  its  linear  models  with  beta  wei^ts,  and/or  the  formal  project 
selection  methods,  must  consider  the  myriad  effects  inherent  in  the  dedsion 
environment  over  time,  across  organizations,  and  through  hierarchical  levels. 

4.7  Question  4:  Effect  of  a  Cost  Estimate 

What  is  the  part-worth  utility  of  an  engineering  development  cost  estimate  in 
a  decision  maker's  choice  on  a  lateral  airfoil  design? 

Section  4.6  above  developed  a  list  of  detemunant  attributes  for  Wright 
Laboratory.  The  same  interview  instrument  that  recorded  participant  ratings  of 
the  utility  of  the  criteria  also  requested  an  assessment  on  the  utilify  of  an  engi¬ 
neering  development  cost  estimate.  Remember  that  the  lateral  airfoil  research 
project  presented  to  the  participants  showed  a  fairly  developed  form. 

Each  subject  was  verbally  requested  to  make  their  evaluation  not  on  the 
dollar  figure,  but  on  the  fact  that  they  had  an  estimate  at  that  juncture.  Amazing¬ 
ly,  this  criterion  received  the  lowest  mean  utility  appraisal  of  all  criteria  rated.  The 
mean  rating  on  a  scale  from  zero  to  150  was  78.3.  (generally,  the  participants  felt 
it  was  more  important  to  have  a  test  plan  than  a  cost  estimate.  Additionally,  the 
Spearman  rank  correlation  failed  to  show  any  significant  correlation  for  the  cost 
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estimate  with  the  other  determinant  attributes  advanced  at  the  .05  significance 
level. 

SAS/STAT  paired  comparison  of  means  procediires  for  the  engineering 
development  cost  estimate  against  the  other  six  1979  criteria  led  to  rcgectimi  of  the 
null  hypothesis  that  the  mean  scores  were  statistically  equal  at  a  =  .05  level  of 
significance.  The  p  value  for  the  tests  ranged  firom  .07  for  the  determinant 
attribute  "Time"  to  0.00  for  "User  Need."  This  test  supports  the  opinion  that 
Wright  Laboratory  has  low  utility  for  an  engineering  development  cost  estimate  as 
a  determinant  attribute. 

Now,  one  needs  to  balance  this  finding  with  other  information  gathered 
during  the  interview  process.  Thirteen  participants  raised  "cost  factors"  as  a 
criterion.  The  cut-off  point  for  criteria  elimination  firom  the  top  seven  listing  was 
arbitrarily  set  at  14  participants.  So,  had  the  list  been  expanded  to  eight  criteria, 
than  cost  factors,  collectively,  would  have  sxurfaced  in  an  important  role.  Mention 
of  these  other  cost  factors  in  the  Wright  Laboratory  interview  sessions  suggests 
that  the  decision  makers  require  a  different  cost  factor(s)  than  the  engineering 
development  cost  estimate  at  this  jimctuie  in  the  decision  process. 

4.8  Chapter  Siimmarv 

This  chapter  detailed  the  results  obtained  fimn  the  siurvey  and  interviews 
conducted  while  exploring  the  specific  issue  ano  answering  the  research  questions. 
It  uses  both  narrative  and  graphic  tools  to  analyze  and  discuss  the  results  obtained 
from  the  data.  The  following  chapter  summarizes  the  key  findings  presented  in 
Chapter  IV  and  lists  specific  recommendations  for  the  Wright  Laborator'  project 
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selection  process  as  well  as  future  research  recommendations  for  other  research 
efforts  including  lateral  airfoils. 
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V.  Conclusions  and  Rft<»omniftndatinn« 


5.1  Introduction 

This  chapter  contributes  an  abridged  precis  of  the  research  findings  and 
recommends  futxure  research.  Generally,  the  inquiry  served  USAF,  DoD,  and 
national  needs  for  description  of  techniques  used  in  effective  resource  allocation. 
Prior  studies  on  Wright  Laboratory  complemented  with  this  research  methodology 
and  results  have  appeared  in  prior  chapters.  Discussions  and  recommendations 
included  in  this  chapter  spring  firom  the  research  questions  which  highli|^ted 
imique  facets  of  the  research  olrjectives.  The  format  includes  recommendations  for 
future  research  on  each  question. 


5.2  Question  1;  Project  Selection  Methods 

What  methods  do  managers  use  to  select  R&D  projects  and  allocate  laboratory 
resources  at  Wright  Laboratories? 

5.2.a  Findings.  The  existence  of  previous  research  by  Prince  (1985) 

and  Congdon  (1988)  provided  a  historical  benchmark  for  imderstanding  Wright 
Laboratory’s  processes  and  tools  while  the  SAS^TAT  software  calculations  of  1993 
survey  results  accurately  measured  the  environment.  Six  important  and  positive 
conclusions  follow  firom  the  research. 
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1.  Wright  Laboratory  uses  many  of  the  published  and  well  documented 
fcomal  prcgect  selection  techniques.  The  study  results  reflect  a  lar^  organization 
adapting  technique  usage  through  the  years,  across  organizational  structure,  and 
down  hierarchical  levels  to  meet  unique  decision  environments. 

2.  Wri^t  Laboratory  displays  a  greater  tendency  to  use  formal  method 
today  than  was  demonstrated  in  Prince’s  1985  or  Congdon’s  1988  study. 

3.  Wright  Laboratory  prefers  using  Checklist,  Scoring,  and  Sorting  modeb 
although  the  survey  identified  a  healthy  familiarity  rate  with  many  other  pub¬ 
lished  decision  making  methods. 

4.  Wright  Laboratory  demonstrates  that  formal  technique  iisage  is  not 
predicated  upon  familiarity.  For  example,  a  large  percent  were  familiar  with  the 
cost/benefit  ratio  method,  but  very  few  used  it. 

5.  Wright  Laboratory  expresses  xmcertainty  on  if  a  formal  project  selection 
technique  could  be  developed  that  would  be  useful  during  the  project  selection  and 
resoiurce  allocation  process,  but  expressed  a  willingness  to  participate  in  a  formal 
project  selection  class  if  offered. 

6.  Wright  Laboratory  shows  a  greater  tendency  to  use  a  formal  decision 
making  technique  as  the  dollar  value  of  the  judgment  increases.  Further,  the 
laboratory  showed  that  formal  methods  have  uses  when  money  is  not  a  factor. 
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5.2.b  RecomiwftwHatioilS.  Specific  recommendations  follow  finom 


survey  and  interview  results.  The  interviews  provided  insight  into  the  directorates 
personalities.  For  example,  survey  statistics  clearly  show  that  Materials  director¬ 
ate  has  a  strong  preference  for  using  formal  decision  making  methods.  Yet, 
statistical  numbers  hide  the  enthusiasm  conveyed  during  personal  interviews. 
Naturally,  a  first  recommendation  follows  firom  an  appreciation  of  the  directorates 
knowledge  of  formal  decision  making  methods  and  statistical  verification  of  an 
overall  ardor  within  the  organization  for  a  deeper  understanding. 

1.  Wright  Laboratory  should  encourage  continued  education  in  decision 
making  methods  appropriate  for  R&D  Project  Selection.  This  could  include 
inter-organization  commimication  (crossfeed/crosstell)  of  formal  method 
experiences  for  both  successes  and  failures. 

A  second  recommendation  targets  postgraduate  educational  institutions  and 
results  from  Wright  Laboratory’s  statistically  large  population  of  Master  degree 
personnel  with  Engineering  majors. 

2.  Educational  institutions  should  encourage  classes  on  formal  decision 
making  methods  at  the  masters  level,  particularly  for  engineers. 

Recommendation  three  has  a  historical  and  statistical  information  basis,  but 
originates  fiom  my  personal  observations  while  tr3ring  to  fathom  the  complexities 
of  published  decision  making  methods.  The  recommendation  build  on  Prince’s 
observation  that  laboratory  personnel  preferred  "relatively  simple  and  easy  to  use" 
techniques  (Prince,  1985:38). 

3.  The  propensity  of  usage  rates  for  simple  techniques  suggests  a  lack  of 
imderstanding  for  the  more  rigorous  methods.  Authors  feuniliar  with  formal 
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project  selection  techniques  should  communicate  thoee  techniques  in  both 
engineering  and  management  vernacular. 

A  final  recommendation  concerns  futiure  research,  noting  that  Wright 
Laboratory’s  decision  making  environment  is,  perhaps,  no  less  complex  than  many 
other  agencies  within  and  without  government.  The  potential  and  consequences  of 
mismanagement  warrant  continued  high  level  interest  and  journalistic  attention 
like  that  displayed  in  institutional  research  (Brooks,  1979;  Prince,  1985;  and 
Congdon,  1988)  and  industry  trade  magazines  (Lavitt,  1993:17). 

4.  Studies  of  R&D  project  selection  technique  iisage  needs  to  be  published 
on  other  Air  Force  and  DoD  laboratories  to  complement  this  study  and  meet 
OTA’s  challenge  of  encoiu*aging  research  performers  to  address  these 
national  needs  (OTA,  1991:43).  Additional  studies  could  focus  on  6.1  level 
decision  processes  at  government  agencies  like  the  Office  of  Scientific 
Research,  and  the  Advanced  Research  Projects  Agency. 

5.3  Question  2;  Lateral  Airfoil  Research  Viabilitv 

What  establishes  lateral  airfoils  as  a  potentially  viable  research  project? 

5.3.a  Findings.  The  abimdance  of  bibhometric  evidence  from  patents 

detailed  in  Appendix  A  establish  lateral  airfoil  technologies’  research  merit.  This 
observation  needs  to  be  balanced  by  interview  respondents’  reaction  to  a  potential 
$210  million  dollar  engineering  development  prpgect.  A  majority  of  participants 
suggested  that  they  required  a  much  deeper  understanding  of  the  technology’s 
fundamental  scientific  underpinning  prior  to  any  wholesale  commitment.  Interest- 
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ingty,  Wright  Laboratory  interview  statistics  show  that  bibliometric  evidence  is  a 
positive  indicator  for  research  potential. 

5«3.b  ReconuiiftTidations*  Basic  research  needs  to  be  done  to 
establish  a  better  \mderstanding  of  lateral  airfoil  potential. 

5,4  Question  3;  Attributes  Used  in  Portfolio  Selection 

What  determinate  attributes  do  decision  makers  use  in  the  final  choice  set 
during  research  prx^ram  portfolio  selection? 

5«4.a  FitiHifigg.  Exploratory  research  suggests  Wright  Laboratory  has 

numerous  important  criteria  for  applied  research  decision  processes.  The  initial 
list  of  60  discrete  items  was  reduced  to  the  seven  popular  attributes  shown  below. 

1.  Does  the  project  have  Technical  Merit 

2.  Has  it  received  positive  Ehcpert  Opinion 

3.  Does  supporting  Basic  Research  data  exist 

4.  Does  it  appear  Feasible 

5.  Does  it  meet  an  Air  Force  Mission 

6.  Do  Altematiye  Technologies  already  exist 

7.  Has  there  been  an  expressed  User  Need 

Some  of  these  determinant  attributes  compare  favorably  with  Stahl  and  Harrell 
(Brooks,  1979)  original  six  while  others  surface  for  the  first  time  in  this  study. 
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5.4.b  ReCOHl'WfiftTidHtionS*  The  1993  interview  results  qualified  as 

an  exploratory  study  on  this  subject.  Further  research  needs  to  be  done  to  estab¬ 
lish  the  relative  merits  of  these  attributes,  leading  to  a  predictive  model.  A  larger, 
follow-on  statistical  sampling  could  exploit  these  popular  decision  attributes.  A 
more  in-depth  study  could  explore  determinant  attribute  interactions  for  the  large 
list  of  30  criteria  that  were  identified  by  laboratory  personnel.  Factor  analysis 
procedures  based  on  experimental  results  could  accurately  determine  discrete 
criteria’s  interactions. 


5.5  Question  4:  Effect  of  Cost  Estimate 

What  is  the  part-worth  utility  of  an  engineering  development  cost  estimate  in 
the  decision  maker’s  choice  on  a  lateral  airfoil  design? 

5.5. a  FiTidin|fs-  This  exploratory  study  foimd  that  the  engineering 

development  cost  estimate  has  little  part-worth  utility  to  Wright  Laboratory 
decision  makers  on  applied  research  projects.  Overall,  the  decision  makers  rated 
this  criterion  with  the  lowest  mean  utility  of  any  criteria  either  mentioned  by  the 
interview  participant  or  presented  for  consideration  by  the  interviewer.  However, 
Wright  Laboratory  decision  makers  do  include  other  cost  factors  as  determinant 
attributes  in  their  decisions  process. 

5.5. b  Recommendations.  Additional  research  needs  to  be  done  to 

establish  what  level  of  decision  maker  uses  an  engineering  development  cost 
estimate.  Additionally,  this  research  could  focus  on  identifying  all  cost  criteria  for 
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discrete  oi^anizational  sub-populations  as  well  as  the  appropriate  user  levels  for 
the  engineering  development  cost  estimate  as  a  determinant  attribute. 

5.6  Chapter  Summary 

This  chapter  completes  the  study  on  Wright  Laboratory’s  current  decision 
making  criteria  and  methods.  A  battery  of  findings  and  recommendations  were 
addressed  for  each  of  the  research  questions.  The  study  examined  familiarity  and 
usage  rates  for  fifteen  published  R&D  project  selection  methods  and  foimd  that  the 
laboratory  displayed  a  greater  tendency  to  use  formal  methods  in  1993  than  was 
shown  in  prior  research.  Fxuther,  the  study  showed  the  organization  exercised 
different  techniques  through  the  years  to  meet  unique  decision  environments.  An 
overall  preference  for  simpler  models  like  Checklist,  Scoring,  and  Sorting  models 
led  to  a  recommendation  that  authors  familiar  with  the  other  techniques  commu¬ 
nicate  them  in  engineering  and  management  vernacular. 

Secondly,  the  study  introduced  a  technological  paradigm,  lateral  airfoils.  A 
bibhometric  search  for  patent  designs  dating  to  1910  suggested  a  sustained  trend 
in  the  technology’s  art  and  application.  A  recommendation  was  advanced  calling 
for  basic  research  initiatives  on  lateral  airfoils  leading  to  a  better  xmderstanding  of 
the  technology’s  potential. 

Lastly,  the  study  used  a  "placebo"  lateral  airfoil  research  project  to  gauge 
Wright  Laboratory’s  decision  making  process  and  identified  thirty  discrete  decision 
making  criteria.  Seven  determinant  attributes  were  distilled  firom  this  large  body 
of  criteria.  The  last  research  initiative  led  to  a  recommendation  for  subsequent 
modeling  of  Wright  Laboratory’s  decision  process  using  the  1993  findings. 


109 


Appendix  A:  Lateral  Airfoil  Bibliometrics 


A.1  Lateral  Airfoil  Technology 

This  appendix  provides  an  appraisal  of  three  applications  for  lateral  airfoil 
technology.  Lateral  airfoils  typify  a  technology  that  the  Air  Force  would  research. 
They  have  an  vinknown  usefullness  and  efficiency  potential.  However,  these 
bibliometrics  firom  United  States  patents  suggests  they  may  have  specific  applica¬ 
tions  on  aerospace  vehicles.  The  patents  serves  as  the  springboard  for  discussion 
of  lateral  airfoil  concept  and  potential. 

Lateral  airfoils  perform  the  same  function  on  an  aerospace  vehicle  as  the 
main  rotor  blades  do  on  a  modem  helicopter.  The  helicopter’s  main  rot  r  blades 
’rotate’  in  a  plane  above  the  fuselage  or  helicopter  body  providing  lift  and  thrust. 
Lateral  airfoils  render  the  same  performance  while  ’revolving’  in  a  cylinder  around 
or  outboard  of  the  aerospace  vehicle’s  fuselage.  Copies  of  certain  applicable  United 
States  patents  are  used  in  this  appendix  to  both  explain  the  concept  and  provide  a 
historical  overview  of  their  technological  maturation  process. 

Precedents  for  legal  patents  began  centuries  ago.  Today,  U.S.  patents 
receive  their  legal  authority  firom  Article  1,  Section  8  of  the  United  States  Consti¬ 
tution  and  the  1790  patent  laws.  U.S.  patents  are  issued  only  after  the  Patent 
Office  is  convienced  that  the  new  idea  works,  but  it  is  important  to  note  that  a 
patent  does  not  establish  airworthiness.  The  lateral  airfoil  designs  shown  in  the 
next  paragraphs  represent  what  many  inventors  considered  as  a  truly  remarkable 
discovery  or  innovation  for  their  time.  Their  patents  captured  both  the  designers 
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dreams  and  best  understanding  of  the  concepts.  This  survey  acknowledges  their 
contribution  to  lateral  airfoil  understanding  while  establishing  a  trend  in  thought 
focused  on  a  promising  design  that  offers  unique  operational  characteristics  and 
enhanced  capabilities  for  aerospace  flight. 

There  are  many  novel  designs  documented  with  the  Federal  Government. 
Every  attempt  has  been  given  to  exhaiistively  survey  all  pertinent  United  States 
Patents,  but  some  one  or  more  patents  may  have  been  overlooked  that  describe  or 
significantly  enhance  knowledge  on  lateral  airfoils.  The  patents  and  descriptions 
presented  here  were  gathered  through  a  patent  search  commissioned  by  the  author 
in  1978  and  through  a  "LEXIS"  data  base  search  using  the  key  phrases:  (rotat!  or 
mov!)+(wing  or  airfoiD+dateral  w/5  airfoil).  Again,  inclusion  of  a  particular  patent 
in  the  following  text  does  not  serve  as  a  airworthness  validation  for  any  particular 
design. 

A.l«a  Kincaimon.  The  first  patent  of  interest  was  an  invention  by 

Leo  Covington  Kincannon  fi?om  Santa  Cruz  coimty  California,  USA  (Kincannon, 

1910).  Kincannon  used  the  lateral  airfoil  idea,  referencing  it  as 

two  or  more  pairs  of  wings,  each  pair  being  composed  of  six  separate  wings.  The  wings  are 
mounted  to  revolve  about  an  axis  and  also  caused  to  be  intermittently  rotated,  so  that  for  a 
portion  of  their  travel,  tiuy  will  be  in  a  plane  parallel  with  the  general  plane  or  direction  of 
motion  of  the  flying  machine.  (Kincannon,  1910:1) 

His  legal  language  is  better  understood  when  coupled  with  the  inventor's  drawings 
of  the  design.  The  next  figure  shows  the  inventor's  sectional  view  which  he  titled 
Fig.  3,  but  appears  in  this  text  as  Figure  47.  This  single  drawing  of  the  flying 
machine  is  stifficient  to  explain  the  lateral  airfoil  application.  Kincannon’s  design 
as  shown  had  two  sets  of  lateral  airfoils  rotating  outboard  of  the  fuselage.  Two 
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Figure  47.  KuKannon  Patent  of  1910  (Kincannon,  1910) 
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other  sets  of  lateral  airfoils  are  removed  for  the  sectional  drawing.  These  sets  of 
wings  were  powered  by  a  series  of  drive  shafts  and  gears  connected  to  an  engine  so 
that  the  wings  revolved.  The  inventor  did  note  that  the  gears  worked  to  "drive  the 
shafts  ...  in  opposite  directions"  which  suggests  that  a  potential  torque  factor  was 
considered  (Kincannon,  1910:2). 

Kincannon  incorporated  a  series  of  complicated  changes  in  angle  of  attack 
for  the  wings  in  their  cylinders  of  revolution.  He  described  the  wings  parallel  with 
the  plane  of  revolution  near  the  fuselage  and  perpendicular  to  the  plane  of  revolu¬ 
tion  outboard  of  the  fuselage.  This  detail  is  plainly  shown  in  his  drawing  with 
wings  labeled  162  and  126.  Evidently,  the  inventor  intended  that  the  wings  push 
enough  air  down,  while  outboard  of  the  fuselage,  to  overcone  weight  and  sustain 
fl^ht.  It  is  unclear  if  the  design  would  sustain  flight  for  any  length  of  time,  but 
the  lateral  airfoils  certainly  would  have  chtimed  the  air. 

While  Kincannon’s  design  lacked  many  of  the  conventional  controls  found  on 
modem  aircraft,  he  did  include  a  propeller  for  thrust  and  a  rudder  for  yaw  control. 
He  successfully  patented  the  first  concept  for  airfoils  revolving  outboard  of  the 
fuselage  to  produce  lift. 

A«l.b  Jones.  Lloyd  Jones  of  Las  Animas,  Colorado  received  a  lateral 

airfoil  patent  on  December  17,  1918.  The  inventor's  drawing  fi*om  the  patent 
application  appears  in  Figure  48.  He  called  his  invention  an  "improved  type  of 
aeroplane  embodying  helicopter  principles  of  construction"  (Jones,  1918:1).  His 
design  used  lateral  airfoils  that  may  have  produced  both  lift  and  thrust.  While  it 
is  clear  that  Jones  understood  the  concept  of  auto-rotation  and  recognized  that  the 
lateral  airfoils  could  stablize  the  aircraft  during  auto-rotation  much  like  the 
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Patented  Dec.  17, 1918. 

5  tNCnV-tHCIT  I. 


helicopter's  main  rotor  baldes,  it  is  unclear  if  the  inventor  intended  for  the  engines 
to  drive  the  lateral  airfoils  during  flight  (Jones,  1918:1).  The  drawing  shows  a 
section  view  taken  through  the  vehicle. 

Jones  used  conventioiaal  control  surfaces  like  the  rudder  and  aileron  to 
"assist  in  controlling  the  lateral  and  vertical  movements  of  the  aeroplane"  and 
forward  and  rearward  elevators  which  could  be  "synchronously  oscillated"  (Jones, 
1918:2).  For  thrust,  he  used  three  propellers  in  his  design,  two  for  propulsion  and 
one  for  braking. 

Jones’  lateral  airfoils  were  helical  and  revolved  around  the  fuselage.  His 
design  allowed  the  wings  to  whirl  around  the  fuselage  as  the  vehicle  moved 
through  the  air.  The  engines  could  be  coupled  to  the  lateral  airfoil,  but  the 
coupling  described  in  detail  was  for  auto-rotation  where  the  airfoils  drove  the 
propellers  and/or  engine.  Interristingly,  Jones  contended,  for  the  possibility  of 
vertical  flight,  if  the  pilot  could  skillfully  operate  the  controls.  Ihs  patent  success¬ 
fully  described  the  lateral  airfoils’  auto-rotational  capability  and  introduced  its’ 
potential  for  vertical  flight  application. 

A«1.C  BurrilL  Elvyn  Fremont  Burrill  of  Berkeley,  California  filed  his 

patent  application  on  10  August  1923.  Nearly  two  years  later  the  United  States 
Patent  Office  granted  his  patent  for  a  "new  and  useful  flying  machine"  (Biirrill, 
1925:1).  He  described  a  machine  with  counter  routating  lateral  airfoils  on  twin 
spools  outboard  of  the  fuselage  (see  Figure  49).  The  design  produced  thrust  which 
coxild  overcome  the  vehicles  weight  and  propell  it  vertically  upward.  Additionally, 
he  described  a  helical  curve  on  the  blades  that  could  produce  axial  thrust  for 
propulsion.  Burrill  vectored  the  thrust  fiem  the  lateral  airfoils  and  manipulated 
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the  engine  speed  to  control  the  lift  vector.  He  \ised  two  engines  to  independently 
drive  the  schrouded  lateral  airfoils. 

Biirrill’s  design  lacked  any  pitch  change  method  for  the  lateral  airfoil’s 
wings  or  blades.  He  envoked  directional  control  by  manipulating  the  center  of 
gravity  using  a  sliding  pilof  s  seat.  A  ^sign  which  truly  let  the  pilot  fly  by  the 
seat  of  his  pants. 

Burrill  used  lateral  airfoils  to  achieve  vertical  flight,  hover,  and  transition 
to  horizontal  flight.  He  described  the  effects  of  variable  wing  or  blade  speed  and 
thrust  vectoring. 


A.l.d.  Watter.  In  1927  Michael  Watter  received  a  patent  for  his 

heavier  than  air  lateral  airfoil  design.  His  document  was  a  refile  for  an  abandoned 
application  first  filed  on  25  April  1922.  This  places  the  original  design  before 
Burrill’s  application,  but  since  the  patent  was  granted  in  1927,  a  date  subsequent 
to  Burrill’s,  its’  discussion  apropriately  fjedls  here  in  the  chronological  sequence. 

Watter’s  concept  used  "one  or  more  planes  or  wings  . . .  mounted  to  move  in 
a  rectilinear  direction"  (Watter,  1927:1).  Sheet  one  drawings  of  the  inventor’s 
design  appear  on  the  following  page  (see  Figure  50)  and  show  a  lateral  airfoil 
which  revolves  in  an  oblong  manner.  The  wings  travel  across  the  upper  surface  of 
the  fuselage,  along  the  fuselage  sides,  and  complete  their  journey  inside  the  "fl3dng 
machine"  (Watters,  1927:3).  His  design,  if  actually  seen  in  flight,  could  have 
appeared  as  a  flattened  cigar  with  both  upper  and  lower  vertical  stabelizers  and 
left  and  right  horizontal  stabilizers.  He  described  a  propeller  and  motor  combina¬ 
tion  at  the  nose  and  tail  of  the  vehicle. 
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Watters  design  used  countra  rotating  lateral  airfoils  which  eliminated 
severe  torque  problems.  He  wrote  that  "as  one  set  moves  in  one  direction  the  other 
set  of  planes  moves  in  the  opposite  direction”  (Watters,  1927:1).  The  set  of  planes 
or  wings  produced  lift  as  air  flowed  over  their  surfaces.  Watters  rigidly  attached 
gxiide  rails  to  the  fuselage  which  transmitted  pitch  chang  information  to  the  lateral 
airfoils.  So  there  was  no  way  to  vary  the  pitch  short  of  remounting  the  guide  rails. 
He  wrote  that  the  lateral  airfoils  could  be  used  by  the  vehicle  for  "rising  or 
descending  vertically,  or  hovering  in  the  air  over  any  desired  spot”  (Watters. 
1927:2).  Additionally,  he  suggested  that  the  lateral  airfoils  placement  on  the 
fuselage  could  be  changed.  He  noted  that  they  could  be  arranged  longitudinally  on 
the  fiiselage. 

Watters  contrihutions  included  extension  of  lateral  airfoil  designs  into 
shapes  other  than  circular.  He  introduced  countra-rotating  lateral  airfoil  sections 
which  eliminated  a  substantial  torque  prd)lem. 

A.l.e  Silver,  in  the  dawn  of  aviation  their  was  a  keen  interest  among 

inventors  about  dirigible  airships.  At  one  time  they  held  great  promise  for  airbom 
battle  statiohs.  However,  interest  faded  in  dirigibles  as  a  result  of  numeroiis  prob¬ 
lems  including  the  dangers  encoimtered  when  using  highly  explosive  hydrogen  gas 
to  distend  the  balloon’s  fabric.  Dirigibles  have  remained  a  topic  for  study  throiigh- 
out  the  years  since  the  highly  publicized  "Hindenberg"  disaster  and  many  fly  today. 

One  of  the  problems  typically  encountered  with  dirigible  designs  was  how  to 
propell  the  lighter  than  air  vehicle  (see  Figure  51).  Without  some  type  of  self 
propulsion  device,  the  dirigible  would  simply  drift  with  the  vdnd  and  be  useless  for 
many  types  of  work.  Jesse  W.  Silver,  of  Tacoma,  Washington  successfully  patented 
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March  18,  1930. 


1,750,765 


J.  W.  SILVER 

raorBUilSG  HBAHS  FOR  DIRISIBLS  AIRSHIPS 

Filed  Feb.  7.  1928  2  Sheets-Sbeet  1 


Figlire  51.  Silver  Patent  of  1930  (SUver,  1930) 
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a  dirigible  propulsion  design  using  lateral  airfoils  which  he  daimed  would 

. .  prepare  a  unit  whereby  head  resistance  to  travel  of  an  airship  is  reduced  to  a 
minimum  so  as  to  make  possible  a  greater  speed"  (Silver,  1930:1). 

He  proposed  using  two  or  more  lateral  airfoils  driven  by  an  engine  through 
belts  and  cables  and  mounted  circumferentially  on  the  ship  at  specified  intervals. 
He  wrote  that  the  vanes  or  wings  would  be  rigidly  fixed  at  an  angle  45  degrees  to 
the  belts  direction  of  movement  (Silver,  1930:2).  He  described  torque  effects  which 
would  cause  the  airship  to  axially  rotate  and  proposed  that  the  lateral  airfoils 
rotate  in  opposite  directions  to  negate  the  undesirable  effect.  He  did  not  discuss 
using  the  lateral  airfoils  for  directional  control  and  considered  roll  control  capabil¬ 
ities  only  to  the  extent  of  overcoming  undesirable  torque.  Silver^s  design  success¬ 
fully  described  lateral  airfoil  application  for  dirigible  airship  propulsion. 

A«l.f  Plirpiira*  August  C.  Purpixra  of  Berwyn,  IL  patented  a  lateral 

airfoil  design  in  1956  which  was  proposed  as  more  efficient  in  vertical  fli^t  than 
contemporary  conventional  helicopter  design  of  the  195(f  s  and  capable  of  200  mph 
forward  velocities  (Purpura,  1956:1).  He  detailed  a  single  lateral  airfoil  which 
revolved  around  the  fuselage  aft  of  the  cockpit  (see  Figure  52).  The  lateral  airfoil 
blades  were  connected  to  a  steel  spring  wire  encased  in  rubber  and  p>owered  by 
either  of  two  engines.  As  the  pilot  moved  a  control  shaft  to  the  right  or  left,  he 
initiated  a  complicated  set  of  movements  through  cams  and  gvddes  to  the  airfoils 
revolving  arovind  the  craft.  The  pilot’s  inputs  moved  a  guide  up  and  down, 
changing  the  pitch  of  the  lateral  airfoil  blades  or  wings,  permitting  the  aviator  to 
"stabilize  the  aircraft  in  flight  exactly  as  he  could  with  ailerons  in  conventional 
airplanes"  (Piupura,  1956:2). 
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The  craft  utilized  a  conventional  rudder  and  elevator  for  yaw  and  pitch 
control.  Two  controllable  pitch  propellers  supplied  thrust.  Interrestingly,  Purpura 
coimtered  the  torque  firom  the  lateral  airfoil  with  the  propellers  that  were  rotated 
in  the  opposite  direction. 

The  design  recognized  an  engine  out  situation.  "Overrunning  clutches” 
would  decouple  the  engine  from  the  propeUers  which  would  allow  the  propellers, 
now  driven  by  the  wind,  to  power  the  lateral  airfoil.  Purpura  did  not  mention  any 
autorotational  effect  on  the  lateral  airfoil  itself 

Piirpiura’s  patent  developed  controllable  pitch  for  lateral  airfoil  blades  or 
wings.  He  described  a  lateral  airfoil  system  that  provided  lift,  but  no  thrust. 
Ftirther,  his  design  described  lateral  airfoil  track  other  than  circular. 

Fischer.  Not  all  Patents  granted  for  lateral  airfoils  originated 

in  the  USA  (see  Figure  53).  One  such  United  States  patent  was  granted  to  Hans 
W.  Fischer  for  a  Driven  Rotor-Wing  System  for  Aircraft  of  Steffisburg,  Switzerland 
on  Janimary  25, 1966.  His  design  used  lateral  airfoils  that  revolved  outboard  of 
the  fuselage  "according  to  the  known  Voith-Schnider  principle"  (Fischer,  1966:1). 
Voith  Schnider  propellers  receive  a  deeper  discussion  in  Section  2.5.C.  Generally, 
they  are  used  on  marine  vehicles  requiring  great  maneuverability  qualities,  like 
under-water  mine  hunters  and  passenger  ferries. 

Fischer  described  lateral  airfoil  blades  with  variable  angle  of  incidence  from 
"zero  to  positive  values  and  back  again  to  negative  values  in  the  cource  of  each  full 
revolution"  (Fischer,  1966:1).  This  action  resulted  in  in  a  constant  engine  r.p.m., 
while  varing  the  thrust  vector’s  direction  and  magnitude.  Further,  the  blades  were 
mounted  "so  that  the  distance  between  their  longitudinal  axes  and  the  rotor  axis" 
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Jan.  25,  1966 


H.  W.  FISCHEK 

DRIVEN  BOTOR-NIMg  STSTSU  FOB  AIBCBAFT 
Filed  March  la  1964 


3^31,220 


Figure  53.  Fischer  Patent  of  1966  (Fisher,  1966) 
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fadlitating  a  variable  lift  vector  (Fischer,  1966:2). 

PischeFs  design  was  considerably  more  astetbically  pleasing  than  any 
previous  lateral  airfoil  design.  He  described  an  aerospace  vehicle  capable  of 
vertical  take  off  or  landing.  His  discussed  the  need  for  variable  angle  of  incidence 
for  the  blades,  but  neglicated  their  contribution  to  pitch,  roll,  and  yaw  control.  He 
described  a  twin  lateral  airfoil  system,  but  did  not  discuss  potential  problems  with 
differential  lift  or  thrust  vectors  &om  the  systems. 

FischeFs  patent  was  the  first  to  envoke  a  documented  scientific  principle, 
(Voith-Schnider)  to  support  the  design.  Further,  he  opened  the  door  for  lateral 
airfoil  applications  in  the  marine  environment. 

A.IJ1  Dell*AnMilla.  American  Patent  laws  allow  inventors  to  asign 

their  patents  to  business  entities.  Joseph  L.  Dell’Aquilla  listed  the  Wendros 
Company,  Hicksville,  N.Y.  as  the  assignee  making  his  the  first  patent  on  lateral 
airfoils  to  do  so  (see  Figure  54).  His  1974  patent  included  13  claims  and  12 
drawing  figures  which  described  a  lateral  airfoil  aerospace  vehicle  "capable  of 
omnidirectional  flight  having  a  plurality  of  blades  or  wings  revolving  outboard  of 
the  fuselage  (Dell’Aquila,  1974:1). 

Dell’Aquila’s  listed  five  specific  objectives,  which  describe  potential  advan¬ 
tages  of  the  lateral  airfoil  design  over  the  conventional  helicopter.  They  are  listed 
on  page  127. 
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1.  A  rotary  WMf  airentft  not  subject  to  gyroseopie,  tnumHoual  and 
peudulum  ^eetsfouad  with  eouveutioHal  keUcopUrs. 

2.  Ah  aerospace  vehkie  capabk  of  vertical  takeoff  or  laadtugfree  of 
retr^aimg  Made  stall  ckaraeleristics,  and  thus  capable  of  greater 
velocities  tiuai  a  coavendonal  heUeopter. 

3.  A  rotary  wing  aerospace  vehkie  with  blades  free  of  to  stress  reversals. 

4.  Ah  aerospace  vehkie  with  vastly  improved  size  and  altitude  timitatioHs 
compart  to  the  eouveutioHal  luScopter. 

5.  Ah  aerospace  vehkk  with  decresed  sensitivity  to  center  of  Ifft  and  center 
of  gravity  stability  probkms  commonly  exereabated  by  high  airspeed  in 
conventional  heUeopter  designs.  (DeU’Aguila,  1974:2) 

Dell’Aquila’s  drawings,  supplied  with  the  patent  and  copied  here,  show  a 
craft  with  four  lateral  airfoil  sections  revdiving  outboard  of  the  fuselage.  His 
patent  described  in  detail  how  the  blade  angles  could  be  varied  to  control  lift  and 
how  changes  in  blade  angle  could  contribute  to  3raw,  pitch,  and  roll  control.  He  de¬ 
scribed  lift  vectors  for  the  lateral  airfoil  sections  both  singlely  and  in  combination, 
as  well  as  for  ascending,  descending,  and  lateral  motion.  He  realized  that  propel¬ 
lers  and/or  propulsion  means  could  be  incorporated  into  the  lateral  airfoil  support 
structure  and  that  the  projieller  could  directly  drive  blades  as  the  aerospace  vehicle 
traveled  forward  through  windmilling  (Dell’Aquila,  1974:5).  Further,  he  described 
both  forward  and  reverse  thrust  from  the  lateral  airfoils. 

Dell’Aquila’s  patent  adveinced  new  lateral  airfoil  concepts  and  imderstand- 
ing  while  repeating  many  of  the  capabilities  discussed  by  earlier  inventors.  His 
patent  discussed  mechanical,  hydraulic  and  electrical  means  for  varying  lateral 
airfoil  blade  angle  using  a  cam  and  cam  follower  (Dell’Aquila,  1974:6).  Additional¬ 
ly,  he  noted  that  lift  was  created  by  the  lateral  airfoil  blades  at  "both  the  top  and 
bottom  of  the  rotational  cycle"  and  described  a  compensation  the  downwash  effect 
(Dell’Aquila,  1974:2). 
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AJ2  ApIH^ndiir  fiiimmarv 


This  appendix  presented  an  a  view  of  lateral  airfoil  patents  granted  for 
aerospace  vehicle  designs  since  1910.  The  bibliometric  evidence  suggests  a 
sustainedi  trend  in  thought  and  application  for  the  concept.  The  patent  search  on 
lateral  airfoil’s  is  not  exhaustive.  Additionally,  other  disciplines  may  incorporate 
this  technology.  Examples  may  include  turbine  engine  designs  and  naval  propul¬ 
sion  applications. 
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Appendix  Hr  RAT>  Prolect  Selection  Technignes 


B.1  Introduction 

The  appendix  partitions  a  very  broad  topic.  The  study  encompasses  R&D 
review  methodologies,  or  decision  making  tools,  used  by  agencies  with  research 
allocation  authority.  Typically,  these  review  processes  may  include  peer  review, 
solicitation  of  outside  expert  advice,  or  pr<^ram  manager  judgment.  Specific 
methodologies  used  by  individuals  or  groups  may  include  quantitative  factors  like 
an  Internal  Rate  of  Return  (IRR)  or  qualitative  factors  like  research  importance, 
spin-off  implications  and  portfolio  variely  or  homogeneity.  The  discussion  includes 
a  limited  examination  of  R&D  prqject  selection  models  currently  in  wogae,  includ¬ 
ing  their  strength,  weaknesses,  and  literary  references.  The  presentation  format 
targets  managers  who  desire  a  grasp  for  the  methodologies  without  a  first  getting  a 
fundamental  understanding  of  their  technicalities.  The  collection  here  does  not 
exist  elsewhere  in  literature.  Economic,  scoring,  constrained  optimization,  and 
decision  theory  models  typify  the  material  discussed.  Please  remember  that  data 
on  this  subject  is  extensive.  The  following  information  serves  to  introduce  the 
subject  and  acquaint  the  reader  with  terms  used.  Baker  and  Pond  (1964);  Cetron, 
Martino,  and  Roepke  (1967);  Souder  (1978);  and  Jackson  (1983)  each  give  far  more 
exhaustive  reviews  of  R&D  project  evaluation  methods  while  Clear,  Lockett  and 
Pearson  (1971)  tailored  their  review  to  the  subset  of  techniques  known  as  R&D 
project  portfolio  methodologies. 


129 


ChecfcHut  A  Seoriny  Models 


Many  authors  consider  these  as  basic,  conceptually  simple  techniq\ies.  Tliey 
are  the  outgrowths  of  post  World  War  II  indtistrial  expansion.  Interestingly,  some 
authors  submit  that  these  techniques  receive  broader  use  nearly  a  half  a  century 
later  while  other,  more  mathematically  rigorous  methods  find  only  narrow  applica¬ 
tion  (liberatore  and  litus,  1983;  Watts  and  Higgins,  1987). 


Checklist  Modek  Many  performance  criteria  are  listed.  Each  project  is 

evaluated  as  either  meeting  or  not  meeting  the  criteria.  A  check  mark  is 
made  when  a  criterion  is  met.  No  check  mark  shows  a  criterion  was  not 
met.  After  successfully  evaluating  all  the  criteria,  each  project  receives  a 
value  equal  to  the  sum  of  its  check  marks.  Alternate  forms  of  this 
technique  may  incorporate  several  individuals’  ratings. 

Strength:  Simple  to  use 

Easy  match  of  criteria  to  available  information 
Eai^y  accommodates  qualitative  data 

Weakness:  Overlooks  problem  complexities 

No  way  to  ensxue  rater  used  proper  consideration 
Overlooks  problem  interrelations 
No  prioritization  of  individual  fectors 
Adversely  affected  by  inaccurate  information 

Readings:  (Ansoff,  1962;  Moore  and  Baker,  1969;  Gaver  and 

Srinivasan,  1972;  Augood,  1973;  Souder,  1975;  Jackson,  1983; 
Souder  and  Mandakovic,  1986) 


Soorii^  Model:  "CheckBsts  or  pntfSe  charts  for  d^ereat  projects  are  d^ficalt  to  compare 

because  individual  criteria  are  not  weight^  Sctmng  models  attempt  to  remedy  ttiis  problem 
by  as^ning  n^hts  to  individuai  criteria  and  summarizing  the  results  as  a  single  project 
score.  To  arrive  at  a  set  of  criteria  weights,  it  is  necessary  to  extract  preference  funedons 
flom  die  decision  makers,"  (Jackson,  1983) 

Strength:  Simple,  easy  to  use 
Flexible 

Easily  developed  standards 

Considers  a  range  of  both  economic  and  noneconomic  data 
Provides  a  single  number  evaluation  for  each  project 
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Weakness:  Considers  only  competing  projects 

Lack  of  mathematical  rigor  promotes  questimiable  results 
Dimensionless  and  useless  for  rank  order  comparisons 
Requires  more  information  than  the  checklist 

Residings:  (Jackson,  1983;  Moore  and  Baker,  1969;  Gibson,  1981; 
Krawiec,  1984;  Roessner,  1985;  Souder  and  Mandakovic, 
1986) 


B.3  Econftiwlf!  Mndcla 


These  models  rank  a  prqject  on  its  potential  to  provide  monetary  benefits 
over  time. 


Cost/Benefit  modek  "Money  nUnodom  is  tke  soul  of  bemfU-cost  analysis:  assessing  the  good 
and  bad  aspects  of  decision  aUemadees  by  valuing  them  in  terms  of  money.  Betufis-cost 
analysis  •  synonymous  with  cost-benefit  analysis  and  abbreviaud  as  B/C  or  CIB  -  uses 
monetary  valnadon  to  adueve  commensurabiSty  of  off  decision  attributes.  An  Mtribnte 
valued  as  equaUvent  to  a  one  doOar  gain  is  usually  considered  to  be  canceled  by  another 
attribuu  valued  as  equaUvent  to  a  one  dollar  loss.  With  this  perspective,  the  dlfferenee 
bedveeu  benefits  (good  attributes)  and  costs  (bad  attribmtes)  is  considered  tobe  dte  objective 
function  in  benefit-cod  analysis."  (Thompson,  1980) 

Strength:  Forces  quantification  of  prqject 

Provides  a  sin^  index  for  prqject  comparison 

Weakness  Not  all  non-economic  factors  can  be  translated  into 
dollars 

Not  a  useful  tool  for  evaluating  alternative  funding  levels 

The  ratio  doesn’t  reflect  any  prqject  risk  elements 

Gives  an  expected  value,  not  a  range  of  values 

Doesn’t  consider  resource  constraints 

Doesn’t  consider  how  the  costs  and  benefits  are  distributed 

Requires  significant  amounts  of  information 

Tends  to  lead  to  status  quo 

Contingent  on  existing  distribution  of  income  and  wealth 
The  opportunity  costs  are  difficult  to  estimate  if  alternative 
uses  are  not  known 

Readings:  (Squire  and  van  der  Tak,  1975;  Irwin,  1978;  Thompson, 
1980;  Jackson,  1983;  Smith,  1986) 
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Paybadi;  period:  "Measans  tke  time  it  wOl  take  te  recoup,  in  form  pf  cash  inflow  from 

Ou  Mai  Mian  imvested  iu  a  project  Tke  papback  autitod  ki^Uigkts  UqukBty, 
which  is  efleu  au  importamt  factor  iu  husiuess  decisious.  Projects  wUk  shorter  paybacks 
(more  UfukO  <vv  preferred  to  projects  with  long  paybacks,  if  aU  other  Mugs  are  equoL" 
(Uorugreu  aud  Foster,  1991:681) 

Strength:  not  afifected  by  accrual  accounting  conventions  like 
depreciation  or  depletion. 

Weakness:  Neglects  profitability  over  lifetime  of  the  investment 
Biased  against  long  lived  prqjects  with  low  initial  3neld 
Neglects  time  value  of  money 
Does  not  average  projects  in  order  of  preference 

Readings:  (Kom  and  Boyd,  1969;  Irwin,  1978;  Homgren  and 
Foster,  1991) 


Net  Present  Value:  This  technique  is  a  discounted  cash  flow  method  of  ''calculating  the 
expected  net  monetary  gain  or  tots  from  a  project  by  discounting  aU  expected  future  cash 
is^ws  and  outflows  to  the  present  point  in  time,  using  required  rate  of  return.  Projects 
frith  higher  net  present  values  are  preferred  to  projects  with  lower  net  present  vtdues,  tf  alt 
otiter  things  are  equal"  (Homgren  tutd  Foster,  1991:675) 

Strength:  The  result  is  shown  in  dollars 

Can  be  used  where  there  is  not  a  constant  required  rate  of 
return  for  each  year 

Weakness:  Does  not  give  sufficient  consideration  to  the  magnitude 
of  dollar  investment  and  the  length  of  the  economic  life  of 
the  project 

Doesn’t  work  well  with  mutually  exclusive  projects 
Ambiguous  about  what  would  happen  without  the  project 
Virtually  impossible  to  calculate  opportunity  costs  since  all 
alternatives  may  not  be  known 

Readings:  (Kom  and  Bqyd,  1969;  Irwin,  1978;  Homgren  and 
Foster,  1991) 


Internal  Rate  of  Return:  The  internal  rate  of  return  is  the  rate  of  interest  at  which  the 
present  value  of  cash  inflows  from  a  project  equal  the  present  values  of  expected  cash 
outflows  of  the  project  IRR  is  sometimes  called  the  time-adjusted  rate  of  return.  Projects 
with  lagher  IRRs  are  preferred  to  projects  with  lower  IRR’s,  if  all  other  things  are  equal" 
(Homgren  and  Foster,  1991:677) 

Strength:  Highly  accurate 

Result  is  a  percentage  value 

Useful  when  comparing  projects  of  different  size 
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Weakness:  Defective  as  a  xneasiire  of  mutually  excl\isive  prcgects 
Needs  a  oonstaiit  rate  of  return  for  eadb  year 
Cannot  add  individual  IRR’s  to  obtain  an  estimate  of  their 
combination  IRR’s. 

Returns  a  single  point  criteria 

Readings:  (Kom  and  B<9d,  1969;  Squire  and  van  der  Tak,  1975; 
Irwin,  1978;  Homgren  and  Foster,  1991) 


B.4  Constrained  QptjmiTntiow  Models 

Typically,  these  models  hig^ght  on  one  or  more  constraints  like  money, 
facilities,  or  work  force.  They  identify  the  best  candidate  through  mathematical 
optimization  techniques. 


Linear  Programming:  This  technique  was  pioneered  by  the  Air  Force.  It 
returns  a  solution  that  attempts  to  maximize  (or  minimize)  a  linear 
function  of  the  decision  variables  where  the  values  of  the  decision  variables 
mmt  satisfy  a  set  of  constraints.  Each  constraint  must  be  expressed  as  an 
equation  or  linear  inequality. 

'TAof  is  the  entire  proNem  eon  be  expressed  in  terms  ef  straight  lines,  planes,  or 
anatogous  geometrical  figures.  There  can  be  no  curved  surface  in  any  graphical  represen- 
tation  of  die  problem.  The  mathematical  model  expressing  the  problem  relams  aU  require¬ 
ments  and  matugemenfs  goals  by  means  of  algebraic  expressions  representing  straight 
lines.”  {Lapin,  1976:208) 

Strength:  It  can  handle  large  problems  with  many  projects  and 
resoimce  constraints 

Supports  easy  assessment  of  assumed  parameter  variation 

Supports  thorough  sensitivity  analysis 

Optimize  benefits  while  recognizing  resomnce  limits 

Weakness:  Requires  a  large  amount  of  information 
Not  adept  in  imcertain  environments 
Not  adept  with  interdependence 
Benefits  must  be  quantified  consistently  with  objective 
function  specification 

Resource  requirements  must  be  clearly  defined 
Limits  on  resoiirce  availability  must  be  identified 
Doesn’t  work  with  nonlinear  relationships 

Readings:  (Gear,  Lockett  and  Pearson,  1971;  Gass,  1975;  Lapin, 
1976;  Asher,  1978;  Anker  and  Tyebgee,  1978;  Markland, 
1983;  Jackson,  1983) 
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Int^per  Pn^ramminip  "Simpty  stated,  am  mteger  prognummii^  probkim  isamLP  im  which 
soime  or  of  the  variables  are  required  to  be  moamegadve  integers."  (Winston,  1991:457) 

Strength:  Supports  fixed  charges,  either/or  constraints,  and  related 
ideas 

Shares  many  of  the  same  strengths  as  linear  Programming 
Easily  incorporates  economic  model  information 

Weakness:  Although  the  feasible  region  is  smaller  than  a  Linear 
Program,  the  typical  integer  Program  is  usually  more 
difficult  to  solve. 

Not  supported  with  efficient  numerical  techniques 

Hard  and  time  consuming 

Large  scale  problems  are  practicably  unsolvable 

Readings:  (Hadley,  1964;  Gass,  1975;  Schrijver,  1986;  Winston, 
1987, 1991) 


Nonlinear  Programming:  A  technique  that  seeks  to  maximize  or  minimize  an 
objective  function.  The  objective  function  or  some  of  the  constraints  may  or  may 
not  be  linear. 


Strength:  Shares  many  strengths  of  Linear  Programming 
Works  well  with  nonlinear  relationships 

Weakness:  Difficult  to  prove  that  the  local  minimums  or  maxim  urns 
are  global 

More  mathematically  difficult  than  Linear  Programming 
Sometimes  the  procedure  returns  only  approximate  optimal 
solutions 

Readings:  (Hadley,  1964;  Gass,  1965;  Winston,  1991) 


D3maniic  Programming:  ". .  ut  technique  that  can  be  used  to  solve  many  optimization 
problems.  In  most  applications.  Dynamic  Programming  obtains  solutions  by  working 
backward  from  the  end  of  a  problem  toward  the  beginning,  thus  breaking  up  a  large 
Unwieldy  problem  into  a  series  of  smaller,  more  tractable  problems.”  (Winston,  1991:715) 

Strength:  Efficiently  handles  probability  of  technical  success 

The  one  computational  technique  that  always  gives  a  global 
optimal  solution  regardless  of  the  number  of  local  optimxmis 
Works  well  with  multiple  period  problems 

Weakness:  Complexities  involved  in  acciurately  determining 

probability  of  success  as  a  function  of  past  and  current  R&D 
spending. 

Only  one  resource  constraint  can  be  considered  at  a  time 
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Can  become  mathematically  intractable  with  prc^iressively 
larger  parameters 

Almost  all  problems  need  individual  specification 
Unlike  Linear  Programming,  there  is  no  algorithm  which 
can  be  applied  to  universally  to  all  problems 

Readings:  (Bellman,  1957;  Hess,  1962;  Hadley,  1964;  Trueman, 

1974;  Jackson,  1983;  Bierman,  Bonini,  and  Hausman,  1986; 
Winston,  1991;  Smith,  1991) 


0-1  Integer  Programming:  A  special  case  of  integer  linear  programming  where 
some  or  all  of  the  variables  are  forced  to  take  on  a  unity  value  (1)  or  null 
value  (0).  Any  Integer  Program  can  be  reformatted  as  an  equalivent  0-1 
Integer  Program.  Texts  on  0-1  Integer  Programming  typically  include  a 
discussion  of  its  abilities  to  solve  capital  budgeting  problems. 

Strength:  Reduces  a  large  ninnber  of  potential  projects  to  a 
Manageable  level 

Works  well  with  capital  budgeting  and  fixed  charge  problems 
Works  well  with  either/or  constraints 

Weakness:  Sometimes  it  proves  unsolvable 

Readings:  (Bush  and  Richardson,  1974;  Fox  and  others,  1984; 
CzackowsM  and  Jones,  19^;  Winston,  1987) 


B.5  Delphi  Technique 

This  techniques  uses  anonymous  judgements  on  a  decision  issue  solicited 
from  more  than  one  independent  source.  Initially,  the  opinions  may  be  gathered 
through  a  questionnaire  or  other  format.  The  participant  responses  are  summa¬ 
rized  and  the  results  are  sent  back  to  the  participants  so  they  can  contribute 
feedback  on  their  peer  group  evaliiations.  This  cycle  may  continue  in  an  iterative 
fashion  until  the  group  reaches  a  satisfactory  conclusion. 

Strength:  No  bias  effects  occurring  from  face-to-face  interaction. 

»  The  participants  don’t  have  to  communicate  with  each  other 

or  be  in  the  same  ph3rsical  location. 

Easy  for  the  experts  to  change  their  mind 

*  Allows  an  individual  to  take  a  position  without  all  the  facts 

Easy  to  contradict  superiors  without  condemnation 
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Good  technique  for  non-analytical  problems 

Useful  when  more  individuals  needed  than  can  effectively 

interact  face  to  face. 

Good  method  to  use  when  time  and  cost  factors  prohibit 
group  meetings. 

Weakness:  Slow  process  driven  by  participant  response  cycle 
No  way  for  participants  to  detect  distortion  of  feedback 
Repeated  use  on  the  same  topic  with  the  same  participants 
leads  to  regurgitation  of  the  same  ideas. 

Exhibits  over-optimism  in  the  short  term  and  over-pessimism 
in  the  long  range. 

The  group  consensus  can  overcome  the  maverick  with  the 
better  idea. 

Participants  may  use  simplistic  misjudgments  on  complex 
issues. 

Readings:  (linstone  and  Turoff,  1975;  Basu,  1977;  Dalkey,  1951; 
Eschenbach  and  Geistauts,  1985;  Parente,  Anderson,  Myer, 
and  O’Brien,  1984;  Nancarrow,  1987) 


B.6  Decision  Theory  Models 

These  techniques  predict  the  overall  success  of  a  sequence  of  interrelated 
projects  or  a  project  requiring  a  long  time  period.  They  include  many  specific 
techniques  like  Decision  Tree,  Utility  Thecay,  Fault  Tree,  and  Relevance  Tree. 
Althou^  each  of  these  specific  techniques  is  not  discussed  in  detail  here,  a 
synopsis  of  strengths  weaknesses  and  readings  for  the  group  are  detailed. 

Strength:  Ability  to  consider  a  wide  range  of  uncertainty 

Forces  consideration  of  all  interrelated  sub-project  impor¬ 
tance 

Yields  a  meaningful  arrangement  of  complex  decision 
elements  without  tableau  format  restrictions 
Communicates  every  course  of  action  and  all  possible 
outcomes 

Displays  interaction  between  present  decision  alternatives, 
uncertain  events,  and  future  choices 
Easily  combines  economic  methods  for  a  clear  portrayal  of 
futiure  alternatives  and  events 
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Weakness:  Reqmres  a  large  amount  of  high  quality  input  data 
Highly  complex  model  which  may  not  be  xmderstood  by  all 
May  not  deal  adequately  with  resoiirce  constraints 
Holds  potential  of  becoming  highly  complex 

Readings:  (Hespos  and  Strassman,  1965;  Raifia,  1968;  Trueman, 
1974;  Lapin,  1976;  Balthasar,  1978;  Jackson,  1983;  Souder 
and  Mandakovic,  1986;  Khorramshahgol  and  Gousty,  1986; 
Souder,  1975;  Jackson,  1983) 


B.7  Multiple  Criterion  Decisiom  Making 

These  methods  are  used  when  the  number  of  alternatives  and  criteria  is 
large  and  the  job  of  choosing  good  alternatives  is  difficult.  The  process  may  result 
in  a  satisficing  answer  as  opposed  to  optimizing  answer.  Some  authors  subdivide 
this  category  into  two  distinct  halves,  multiattribute  decision  analysis  and  multiple 
criteria  optimization.  Given  this  division,  multiattribute  decision  analysis  is  most 
applicable  to  problems  fi:amed  in  a  context  of  an  uncertain  environment  with 
limited  alternatives.  Multiple  criteria  optimization  works  well  with  deterministic 
problems  with  larger  numbers  of  feasible  alternate  solutions. 

It  is  readily  evident  that  this  grouping  includes  many  unique  techniques. 
The  discussion  is  limited  to  Goal  Programming,  Interactive  Approach,  Compromise 
Programming,  ELECTRE  approach.  Parametric  approach,  De  Novo  Programming, 
Graphic  techniques,  and  Analytical  Hierarchy  Process  (AHP). 


Goal  Programming:  "As  originally  conceived,  it  is  the  attempt  to  minimize  the  set  of 

deviations  from  prespecified  multiple  goals,  which  are  considered  simultaneously  but  are 
weighted  according  to  their  importance.  In  some  cases,  however,  the  name  "goal  program¬ 
ming"  has  been  applied  to  a  procedure  which  is  actually  a  special  case  of  lexicographical 
screening,  as  described  by  Hobbs  (1978).  That  procedure  first  determines  the  alternatives 
that  minimize  the  deviation  of  the  most  important  objective  from  Us  corresponding  goal 
value.  From  those  alternatives  are  chosen  tite  ones  that  minimize  the  deviation  of  the  second 
most  in^rtant  objective,  from  its  goal,  and  so  on  until  all  objectives  and  their  goals  have 
been  considered. "  (Zeleney,  1982:281-2) 
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Strength:  Provides  an  acceptably  good  solution  to  multiple  objec¬ 
tives 

Works  well  with  multiple  incommensiirable  goals 

Weakness:  Requires  objective  function  to  be  linear 

Results  dependent  upon  bow  well  goals  articulated 
Specifying  a-piioii  weights  may  be  difficult 
Time  consuming:  requires  decision  maker  role  in  formulation, 
solution,  and  evaluation 

Readings:  (Kombluth,  1973;  Naussbaum,  1980;  Zeleney,  1982; 
Markland,  1983;  Hannan,  1984;  Seo  and  Sakawa,  1988; 
Tabucanon,  1988) 


Interactive  Approach:  These  techniques  assume  an  existence  (known  or 
unknown)  of  an  underlying  preference  function.  Tabucanon  wording 
captures  the  technique  as  follows: 

"Interactive  methods  usually  consist  of  a  DM  (Decision  Maker)  -  analyst  or  DM-computer 
dialogue.  At  each  iteration,  the  decision  maker  is  asked  about  some  tradeoff  or  prfference 
information  based  on  the  current  solution  in  order  to  determine  a  new  solution.  The  method 
essentially  assumes  that  the  decision  maker  is  unable  to  indicate  "a  priori"  preference 
information  due  to  the  complexity  of  the  problem,  but  that  he  is  able  to  give  preference 
information  on  a  local  level  with  respect  to  a  particular  solution. "  (Tabucanon,  1988:72) 

Strength:  Managers  can  learn  firom  the  interactive  process 
The  best  MCDM  for  qualitative  criteria 

Weakness:  Very  involved  process  for  busy  managers 

Readings:  (Tabucanon,  1988;  Malakooti  and  Deviprasad,  1989; 

Venugopal  and  Narendran,  1990;  Reeves  and  Gonzalez,  1989) 


Compromising  Programming:  "Compromise  is  a  natural  and  necessary  outcome  of 

making  decisions  based  on  multiple,  often  conflicting  criteria.  Ideally,  the  decision  maker 
would  like  to  completely  satisfy  all  criteria,  but  that  may  be  impossible  in  a  practical  sense. 
It  becomes  necessary  to  "compromise"  some  or  all  criteria  to  some  degree  while  attempting 
to  emulate  the  ideal  elution  as  closely  as  possible.  Compromise  programming  attempts  to 
evaluate  various  solutions  based  on  their  mathematical  difference,  or  "distance,"  from  the 
ideal  solution.  The  best  solution  is  one  which  either  minimizes  this  distance  from  the  ideal 
solution,  or  which  maximizes  the  distance  from  the  "anti-ideal"  solution."  (Corbett, 
1986:17) 

Strength:  Solution  is  the  best  of  all  available  solutions 
Designed  for  multiple  incommensurable  goals 

Weakness:  Results  dependent  upon  accuracy  of  a-priori 
weights 
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Requires  articulation  of  all  characteristics  used  for  comparing 
alternatives 

Readings:  (Tabucanon,  1988;  Zeleney,  1982;  Gearhart,  1984; 
Corbett,  1986) 


ELECTRE  Approaches:  ELECTRE  1.  Basically,  this  approach  selects  a  pre¬ 
ferred  alternative  which  meets  most  of  the  criteria  while  minimizing  the 
discontent  from  any  one  criterion.  The  results  appear  in  a  preference  graph 
with  a  partial  ordering  of  alternative  systems. 

ELECTRE  II.  This  results  in  a  complete  alternative  ordering 
using  the  concept  of  strong  and  weak  ranking  relationships.  It  requires  an 
imderstanding  of  the  concepts  of  high,  average,  and  low  concordance,  and 
high  and  average  discordance. 

Strength:  The  method  handles  nonquantifiable  criteria  well 

It  is  responsive  to  the  preference  scale  of  the  decision  maker 
Requires  only  a  priori  preference  articulation  at  a  local  level 

Weakness:  Large  set  of  a  priori  weights  and  thresholds 
(ELECTRE  II) 

Some  concepts  and  quantification  procedures  may  not  be 
appealing  to  the  decision  maker 

The  preference  graphs  become  complicated  and  difficult  to 
interpret  for  larger  numbers  of  alternatives 

Readings:  (Tabucanon,  1988) 

Parametric  Approach:  This  technique  uses  mathematical  or  heuristic  methods 
to  eliminate  several  of  a  large  amount  of  alternative  solutions. 

Strength:  Doesn’t  require  decision  maker  to  articulate  preference 
information 

Decision  Maker  evaluates  only  a  subset  of  satisfactory 
solutions 

Weakness:  May  be  time  consuming  if  many  efficient 
solutions  are  generated 

Readings:  (Tabucanon,  1988) 


De  Novo  Programming;  This  technique  is  discussed  in  detail  by  Zeleney 

(Zeleney,  1976,  1982,  1986).  He  proposes  the  design  of  an  optimum  system 
as  opposed  to  optimizing  a  given  system.  It  uses  a  systems  approach  to 
e  valuate  alternatives  besides  those  that  already  exist. 
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strength:  FxiUy  iises  resoiurces,  no  waste  or  slack 

Uses  a  budget  constraint  as  an  upper  limit  of  activity 
Achieves  a  high  optimum  in  cases  of  fixed  budgets 
Expresses  resource  infeasiblity  in  monetary  terms 
No  need  to  reevaluate  for  impacts  of  marginal  changes  in 
individual  resources 

Weakness:  Not  effective  when  decision  maker  does  not  want  to 
design  a  new  optimal  system 

Readings:  (Tabucanon,  1988) 


Graphic  Techniques:  These  are  visually  interactive  models  that  symboUcally 
depict  expected  outcomes  of  each  alternative  relative  to  each  attribute. 

Strei^th:  Intuitive  by  design 

Present  much  information  quickly 
It  can  incorporate  economic  models 
Presents  a  hohstic  view  of  the  problem 

Weakness:  Analysts  may  agree  on  some  criteria  like  measures 

effectiveness,  but  disagree  on  axis  and  scale  for  output  and 
output  interpretation 

May  require  large  amounts  of  time  for  model  formulation  and 
display 

Readings:  (Cook  and  Sieford,  1982;  Canada  and  Sullivan,  1989; 

Kasanen,  Ostermark,  and  Zeleney,  1988;  Belton  and  Vickers, 
1988;  Keeney  and  Raiffa,  1976;  Graves  and  Ringuest,  1991; 
liberatore  and  Titus,  1983;  Jordan,  1992) 


Analytical  Bfierarchy  Process  (AHP):  A  procediure  that  ranks  alternatives  by 
pairwise  comparison  with  respect  to  a  hi^er  level  goal  or  criterion. 

Strength:  Partitions  large  problems  into  smaller  parts 

Detailed  procedure  to  check  consistency  of  decision  makers 
performance 

Easily  incorporates  tangible  and  intangible  criteria 
Does  not  require  numerical  data 

Weakness:  Pairwise  comparisons  can  become  ambiguoiis 
Pairwise  comparisons  can  become  tedious 
Excessively  large  nximbers  of  pairwise  comparisons  may  be 
required  by  reasonably  sized  problems 
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Readings:  (Saaty,  1980, 1982;  Bard,  1986;  Zahedi,  1986;  Vargas, 
1990;  Boucher  and  MacStravic,  1991;  Battin  and  Bender, 
1992;  Jordan,  1992) 


B.8  Miiltiattribute  Utility  Theory  (MAUT) 

This  technique  uses  advanced  mathematics  with  economic  manipiilation  to 
evaluate  alternatives  against  multiple  criteria.  Zeleney  sums  up  the  process  as 
follows: 

"Multiattribute  utility  theory  arises  from  the  classical  precepts  of  perfect  rationality,  utility  or 
profit  maximization,  and  predietalullity  of  aggregate  phenomena.  MAUT  is  prescriptive, 
concerned  witii  the  choice  among  prespecffied  attematives  according  to  the  principle  of 
Maximization  of  subjective  expected  utility."  (Zeleney,  1982:437) 

The  process  arrives  at  an  optimum  course  to  achieve  the  stated  goal.  It  can  be 

characterized  by  the  following  five  steps: 

1)  Determine  the  goal,  objectives,  criteria,  and  alternatives 

2)  Determine  the  preference  of  each  component 

3)  Assign  quantified  values  to  each  attribute 

4)  Determine  a  value  fimction  that  is  the  sum  of  the  attribute 
values  of  each  alternative 

5)  Perform  a  tradeoff  by  deciding  how  much  of  one  objective  to  give 
up  to  improve  another  objective 

Strength:  Works  well  with  qualitative  and  quantitative  data 

Weakness:  Reqmres  a  competent  analyst 

Requires  reasonably  accurate  numerical  data 

Managers  may  not  imderstand  how  qualitative  data  is 

quantified 

Highly  subjective 

No  sensitivity  analysis  means 

Difficult  to  arrive  at  a  set  of  assumptions  that  simplify  the 
MAUT  fimction 

Assumes  every  decision  maker  is  qualified  to  evaluate  every 
attribute 
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Readings:  (Keeney  and  Raifia,  1976;  Madey  and  Dean,  1885; 
Bouyssou,  1988;  Canada  and  Sullivan,  1989) 
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AppftndiY  r?  lateral  Airfoil  Design  Detail 


C.l  Introduction 

This  appendix  introduces  a  t3rpical  lateral  airfoil  design  for  an  aerospace 
vehicle.  The  narrative  asstunes  a  basic  tmderstanding  of  helicopter  design. 
However,  the  details  discussed  include  elementary  concepts  only.  The  format 
focuses  on  aerodynamic  forces,  fli^t  controls,  high  speed  flight,  autorotation,  and 
munitions  storage. 

CJ2  Forces 

The  aerodynamic  forces  acting  upon  this  new  type  of  aerospace  vehicle  are 
identical  to  those  acting  upon  all  aircraft  A  discussion  of  these  forces  is  elemen¬ 
tary.  However,  it  facilitates  a  better  xmderstanding  of  the  design  as  a  whole. 

During  flight  in  a  no-wind  condition,  the  lift  force  acts  vertically  upward,  the 
weight  force  acts  vertically  downward,  the  thrust  force  acts  horizontal  to  the 
longitudinal  axis  toward  the  nose  of  the  design,  and  drag  acts  horizontal  to  the 
longitudinal  axis  toward  the  tail  of  the  design.  This  is  consistent  with  conven¬ 
tional  aircraft  designs.  As  lift  exceeds  weight,  the  design  climbs;  if  lift  is  less  than 
weight,  the  design  descends.  If  thrust  exceeds  drag,  the  design  speeds  up;  if  thrust 
is  less  than  drag,  it  slows  down. 

A  noticeable  disparity  with  conventional  aircraft  becomes  evident  with  lateral 
movements.  In  sideways  flight,  the  lift  vector  resolves  into  two  components  -  lift 
acting  vertically  upward,  and  lift  acting  perpendicular  to  the  longitudinal  axis. 

The  upward  lift  component  must  exactly  offset  the  weight;  otherwise,  vertical 
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movements  occur.  Also,  as  the  sideways  component  of  lift  increases  the  design 
moves  in  that  direction.  If  the  sideways  lift  component  becomes  less  the 
si^ways  drag,  the  aerospace  vehicle  slows  its  sideways  flight 

Reversing  the  thrust  allows  the  vehicle  to  backup  in  flight.  Thrust  derives 
firom  two  sources.  The  first,  and  most  conventional,  is  the  engine  exhaust.  A 
secondary  source  of  thrust  is  the  lateral  airfoil  exhaust  itself,  which  is  analogous  to 
and  treated  like  bypass  air  firom  a  turbo-£an  engine.  Please  note  Figure  5,  an 
artist  conception  of  the  aerospace  vehicle  in  flight. 

C.3  Flight  Controls 

The  typical  cockpit  flight  controls  parallel  those  of  the  helicopters.  Further, 
pilot  inputs  at  the  controls  yield  comparable  results.  The  similarity  ends  here  as 
many  differences  exist  in  the  way  the  aircraft  interprets  the  control  inputs.  A 
typical  mechanical  linkage  drawing  showing  the  pitch,  yaw  and  collective  cams  is 
given  in  Figure  55.  Other  methods  of  effecting  control  inputs  are  just  as  acceptable 
as  mechanical  linkages  and  perhaps  preferable.  They  include  hydraulic,  pneumat¬ 
ic,  fly-by-wire  and  fly-by-Ught. 

The  cockpit  flight  controls  include  the  cyclic,  collective  and  rudder  pedals.  As 
in  the  helicopter,  the  cyclic  controls  pitch  and  roll;  movement  about  the  vertical 
axis  or  yaw  is  controlled  by  the  rudder  pedals;  and  the  collective  controls  move¬ 
ment  along  the  vertical  axis. 

Moving  the  cyclic  to  the  right  or  left  effects  roll  or  movement  around  the 
longitudinal  axis.  Roll  is  a  complex  operation  accomplished  by  increasing  the 
RPM,  or  torque,  on  one  lateral  airfoil  system  (LAS)  while  decreasing  the  RPM  on 
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the  other  LAS.  Since  the  LAS  mth  the  increased  RPM  will  inoduce  greater  lift,  a 
compensation  in  individual  blade  pitch  must  be  made.  A  converse  actjustment  in 
the  blade  pitch  of  the  slower  moving  LAS  may  be  made  as  well. 

Movement  about  the  lateral  axis  produces  a  nose-up  or  nose-down  attitude. 
The  change  is  effected  by  moving  the  cyclic  pitch  control  fore  and  aft.  Typically,  a 
forward  movement  of  the  cyclic  produces  a  nose-down  pitch.  Remembering  the 
effects  of  gyroscopic  precession  and  referring  to  Figure  56;  linkage  firom  the  cychc 
to  the  forward  LAS  pitch  cam  forces  increased  |ntch  of  the  blades  at  "A,"  while 
those  at  "C"  decrease  pitch.  This  action  pulls  the  nose  down.  Simiiltaneous  to  this 
action,  linkage  connected  to  the  aft  LAS  pitdi  cam  causes  the  blades  at  "E"  to 
increase  pitch  while  those  at  "G"  decrease  intch.  This  action  pulls  the  aft  of  the 
design  up.  The  combined  effects  of  these  complex  interactions  forces  pitch  about 
the  lateral  axis. 

The  collective  pitch  control  varies  the  vertical  lift  produced  by  the  twin-comp¬ 
onent  lateral  airfoil  system  (TCLAS).  The  TCLAS  is  simply  two  LAS’s  located  one 
behind  the  other,  linkage  firom  the  collective  to  the  collective  lift  ^ain  increases  or 
decreases  the  blade  pitch  at  "A,"  "C,"  "E,"  and  ”G."  Raising  the  collective  pitch 
control  increases  the  pitch  of  the  blades  near  "C  and  "E”  while  decreasing  the 
blade  pitch  near  "A"  and  "G."  Lowering  the  collective  forces  an  equal,  but  opposite 
action. 

Many  factors  determine  the  amount  of  lift  available  for  operation.  Generally, 
the  pilot  has  control  over  two  of  these.  One  is  pitch  angle  of  the  individual  LAS 
blades;  the  other  is  RPM  of  the  TCLAS.  Control  of  the  pitch  angle  of  the  blade 
during  its  revolution  around  the  longitudinal  axis  lets  the  pilot  establish  the  flight 
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characteristics  of  the  design.  Manipulatu)n  of  the  throttle  (located  on  the  end  of 
the  collective)  produces  a  constant  engine  RPM  despite  the  increased  or  decrease  in 
blade  pitch.  Furthermore,  synchronization  of  the  throttle  and  the  collective  pitch 
control  increases  the  power  available  with  increased  blade  pitch  and  produces  a 
greater  load  on  the  engine.  Also,  the  engine  power  decreases  with  a  decrease  in 
pitch  of  the  blades. 

Movement  about  the  vertical  axis  produces  yaw;  a  nose  swing  to  the  ri^t  or 
left.  The  directional  control  pedals  (rudder  pedals)  control  this  movement.  The 
pedals  link  to  the  yaw  cam.  The  yaw  cam  controls  the  blade  pitch  at  "B,"  D,"  "F," 
and  "H."  If  the  left  pedal  is  pushed,  the  yaw  of  the  forward  LAS  will  cause 

the  blades  near  ”D”  to  increase  pitch,  while  the  blades  near  "B"  decrease  pitch. 

The  pitch  increases  on  one  side  combined  with  the  pitch  decrease  on  the  other  side 
piills  the  nose  of  the  design  to  the  left.  Simultaneous  to  this  action,  the  rear  LAS 
yaw  cam  will  cause  the  blade  pitch  to  increase  at  "H”  and  decrease  at  "F."  The  aft 
of  the  design  pulls  to  the  right.  The  combined  effect  of  these  actions  produces  yaw. 

C.4  Hiyh  Speed  Flight 

When  hovering  in  still  air  the  relative  wind  remains  parallel  with  the  chord 
line  of  the  TOLAS  blades.  However,  diuing  forward  flight  at  increasingly  higher 
speeds  the  relative  wind  shifts  from  parallel  to  the  chord  line  to  perpendicular  to 
the  chord  line  of  all  the  TOLAS  blades.  The  resultant  of  this  condition  is  a  blade 
stall.  Therefore,  aerospace  vehicles  operating  at  consistently  relative  hi^  forward 
velocities  have  blades  that  are  twisted  as  shown  in  Figure  57.  For  increasingly 
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higher  forward  speeds  the  TCLAS  blade  pitch  completely  flattens  and  more 
conventional  control  surfaces  provide  aircraft  control. 

Autorotation 

Autorotation  is  the  term  iised  when  the  TCLAS  is  driven  by  the  action  of  the 
relative  wind  as  opposed  to  the  engine.  The  transmission,  or  power  train,  disen¬ 
gages  automatically  &om  the  TCLAS  when  the  engine  stops,  allowing  the  TCLAS 
to  revolve  freely  around  the  longitudinal  axis.  Autorotation  technique  and  theory 
remain  identical  to  that  of  the  helicopter. 

C.fi  Munitions 

Conventional  winged  surfaces,  sponson’s  or  a  rotary  type  launcher  (see 
Figure  56)  provides  attachment  points  for  munitions  or  pods.  A  rotary  launcher 
simplifies  easy  access  by  ground  crews  for  installation  of  munitions.  As  they  are 
attached,  the  rotary  launcher  advances  around  the  aerospace  vehicle  allowing  the 
ground  crew  to  attach  the  next  munition  or  pod  from  the  same  groimd  location. 

C.7  Conclusion 

This  concludes  a  brief  description  of  the  twin-component  lateral  airfinl  S3rstem 
and  its  application  to  an  aerospace  vehicle.  Derivatives  of  this  design  could 
provide  novel  gas  trurbine  engines  that  are  more  easily  adapted  to  high  Mach  flight 
conditions,  as  well  as  novel  seagoing  vessels  propelled,  supported  and  controlled  in 
the  water  with  the  same  structural  members.  This  text  and  referenced  drawings, 
while  not  totally  accurate  nor  detailed  enough  to  meet  engineering  specifications, 
serves  as  a  catalyst  for  further  research  into  the  subject. 
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Appftfidiy  D;  R&D  Project  Selection  Survey  Tnstmiment 


D.l  Cover  Letter 


DEPARTMENT  OF  THE  AIR  FORCE 

WMQMT  iASOMATORV  (Amo 
WmOHr^ATIlflSOM  AM  FOACC  BASE.  OMK> 


02  JUt6S3 

FROM:  WL/CC  Bldg  45 

2130  Eighth  St  Ste  11 
Wright-Patterson  AFB  OH  45433-7552 

SUBJ:  RSeO  Project  Selection  Survey 

TO:  See  Distribution 

1.  I’m  reguesting  your  participation  in  a  survey.  The  attachcKi 
survey  instrument  investigates  RW3  project  selection  and  resoxurce 
allocation  at  Wright  Laboratory.  Publication  of  the  discrete  and 
aggregate  R&D  comparison  methodology  used  at  Wright  Laboratory 
may  serve  as  a  model  for  other  leQsoratories  inside  and  outside 
the  USAF. 

2 .  This  Air  Force  Institute  of  Technology  reseeurch  measures  your 
opinions  concerning  the  utility  of  various  formal  decision  making 
techniques.  Besides  the  survey,  some  of  you  may  be  eisked  to 
participate  in  a  short  interview.  I  have  been  assured  of  the 
confidentiality  of  your  responses  in  both  cases. 

3.  If  you  have  any  questions,  or  if  you  would  like  to  obtain  a 
copy  of  the  research  results,  please  contact  the  researcher 
through  the  following  address: 

Department  of  the  Air  Force 
AFIT/LAA  (Capt  iT^mes  Beurger) 

2950  P  Street 

Wright-Patterson  Are  OH  45433-7765 
Office  Phone:  (513)  255-8989 

.UAVID  A.  HERRELKO 
Colonel,  USAF 
Commander 


1  Atch 

R&D  Project  Selection  Survey 
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D.2  Survey 


R&D  Project  Selection  Survey 


Suspense:  Please  accomplish  before  20  June  1993. 

Time:  The  survey  takes  approximately  8  minutes  to  complete. 

Purpose:  The  stirvey  investigates  the  R&D  project  selection  and  resource 
allocation  process  at  Wright  Laboratory.  Several  formal  techniques  have  been 
developed  or  proposed  in  literature  to  help  R&D  managers  in  making  project 
selection  decisions.  This  survey  measures  your  opirdons  concerning  the  utility  of 
these  techniques  and  characterizes  the  resource  allocation  process.  Ptiblication  of 
Wright  Laboratory’s  discrete  and  aggregate  comparison  methodology  is  important 
since  it  may  serve  as  a  model  for  other  institutions  within  and  without  the  USAF. 

A  formal  project  selection  technique  is  any  model  or  algorithm  used  to  systemati¬ 
cally  assign  values  to  individual  R&D  projects  or  groups  of  projects.  Formal 
project  selection  techniques  can  be  grouped  into  the  following  categories: 

1)  subjective  methods;  2)  economic  techniques;  3)  risk  analysis  techniques;  and 
4)  mathematical  programming  models.  A  partial  list  of  project  selection  tech¬ 
niques  appears  in  questions  1  through  15  of  this  survey. 

Confidentiality:  Your  responses  to  the  survey  will  be  .strictly  confidential. 

Survey  Responses:  This  is  a  voluntary  exercise,  however,  you  are  an  important 
part  of  this  survey.  Your  response  is  crucial  to  the  project’s  success. 

None  of  the  survey  questions  require  yoa  to  look  up  information.  Only  your 
opinions  are  sought.  You  should  mark  your  response  quickly  after  carefully 
reading  the  questions.  If  you  would  like  to  make  any  comments,  please  feel  Gree  to 
mark  them  directly  on  the  survey. 

After  completing  this  survey,  please  place  it  in  the  return  envelope  provided  and 
mail  it  promptly.  If  you  have  any  questions  or  you  would  like  to  obtain  a  copy  of 
the  results,  please  contact  me  at  the  following  address: 

Department  of  the  Air  Force 
AFrr/LAA  (Capt  James  Barger) 

2950  P  Street 

Wright-Patterson  AFB,  OH  45433-7765 
Office  Phone:  (513)-255-8989 


THANK  YOU  FOR  YOUR  TIME  AND  COOPERATION. 
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Parti 


The  following  is  a  list  of  formal  project  selection  techniques.  If  you 
are  familiar  with  the  technique,  please  place  an  "S'  in  the  appropriate 
column.  If  you  also  use  the  technique  regularly  in  your  project  selection 
process,  please  mark  the  appropriate  column.  If  you  are  not  fomiliar 
with  the  technique,  please  do  not  mark  either  column. 

Technique  Familiar  Use 

With  Regularly 

1.  Sort  _  _ 

2.  Checklists  _  _ 

3.  Scoring  Models  _  _ 

4.  Delphi  _  _ 

5.  Cost/Benefit  Ratios  _  _ 

6.  Payback  Period  _  _ 

7.  Net  Present  Value/  _  _ 

Internal  Rate  of  Return 

8.  Portfolio  Models  _  _ _ 

9.  Risk  Analysis  _  _ 

(Monte  Carlo  Simulation) 

10.  Decision  Analysis/  _  _ _ 

Decision  Trees 

11.  Linear  Programming  _  _ 

12.  Integer  Programming  _  _ 

13.  Nonlinear  Programming  _  _ _ 

14.  Dynamic  Programming  _  _ 

15.  Goal  Programming  _  _ 

16.  Others?  Please  list 
below. 
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For  the  following  questions,  please  indicate  the  expression  that  most 
closely  matches  your  opinion.  The  answers  are  categorized  using  a  scale 
from  one  to  seven.  Ihe  use  of  the  term  "formal  project  selection 
technique^'  refers  to  techniques  such  as  those  listed  in  questions  1 
throuqh  16. 


17.  How  would  you  assess  the  amount  of  influence  that  you  have  on  your  organization’s 
R&D  project  selection  and  resource  allocation  process? 


Very  Strong  Some  Little  No 

Strong  Influence  Influence  Influence  Influence 

Influence 


1 


18.  How  would  you  assess  the  adequacy  of  the  manner  in  which  your  division  reviews 
and  evaluates  potential  R&D  proiects? 


Could  be 
Improved  a 
Great  Deal 


Could  be 

Improved 

Somewhat 


Uncertain 


Probably 
Could  Not 
be  Improved 


Could 
Not  be 
Improved 


19.  How  would  you  assess  the  potential  worth,  to  you  personally,  of  a  technique  which 
would  be  designed  to  assist  you  in  measiuing  the  comparative  value  of  potential  R&D 
projects? 


Extremely 

Bene¬ 

ficial 


'I 

Somewhat 

Bene¬ 

ficial 


No 

Benefit  or 
Uncertain 


Somewhat 

Detri¬ 

mental 


Extremely 

Detri¬ 

mental 
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20.  How  would  you  assess  the  potential  worth,  to  those  people  with  whom  vou  work,  of  a 
technique  which  would  be  designed  to  assist  them  in  measuring  the  comparative  value  of 
potential  R&D  projects? 


Extremely 

Bene¬ 

ficial 


Somewhat 

Bene¬ 

ficial 


No 

Benefit  or 
Uncertain 


Somewhat 

Detri¬ 

mental 


Extremely 

Detri¬ 

mental 


21.  How  important,  to  you  personally,  would  you  consider  a  decision  to  use  a  formal 
project  selection  technique  on  aU  your  R&D  projects? 


Very 

Somewhat 

Neither 

Somewhat 

Very 

Important 

Important 

Important 

Unimportant 

Unimportant 

to  you 

to  you 

nor  Unim- 

to  you 

to  you 

portant 


22.  If  such  a  decision  were  to  be  made  what  would  you  recommend? 


Definitely  Probably  Uncertain  Probably  Definitely 

Should  Should  Should  Should 


Use 


Use 


Not  Use 


Not  Use 


23.  How  important,  to  vou  personally,  would  you  consider  a  decision  to  use  a  formal 
project  selection  technique  on  at  least  50  percent  of  your  R&D  projects? 


Very 

Somewhat 

Neither 

Somewhat 

Very 

Important 

Important 

Important 

Unimportant 

Unimportant 

to  you 

to  you 

nor  Unim- 

to  you 

to  you 

portant 


1 


2 


3 


4 


5 


6 


7 
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24.  If  such  a  decision  were  to  be  made  what  would  you  recommend? 


Definitely 

Should 

Use 


Probably 

Should 

Use 


Uncertain 


Probably 
Should 
Not  Use 


Definitely 
Should 
Not  Use 


1 

2 

CO 

4 

5 

6 

7 

25.  Do  you  feel  it  is  likely  that  a  formal  project  selection  technique  could  be  developed 
which  would  be  useful  to  you  during  your  project  selection  and  resource  allocation 
process? 


Very 

Likely 


Likely 


Uncertain 


Unlikely 


Very 

Unlikely 


26.  To  what  extent  do  you  feel  "pressure"  from  outside  Wright  Laboratory  to 
quantitatively  justify  yoxu:  R&D  expenditures? 


Extreme 

Pressure 


Significant 

Pressure 


Some 

Pressure 


Little 

Pressure 


No 

Pressure 


27.  To  what  extent  do  you  feel  "pressure"  from  within  Wright  Laboratory  to 
quantitatively  justify  yo\ir  R&IXexpenditiues? 


Extreme  Significant  Some  Little  No 

Pressure  Pressure  Pressure  Pressure  Presstire 
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28.  In  the  space  below,  please  describe  any  positive  or  negative  experiences  you’ve  had 
when  using,  or  attempting  to  use,  a  formal  project  selection  technique. 
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Part  n 


Each  item  in  this  portion  of  the  survey  consists  of  a  statement  with  which 
you  are  asked  to  agree  or  disagree.  Using  the  scale  shown  below,  please 
indicate  the  extent  to  which  you  agree  or  disagree  with  each  statement. 
Place  a  number  which  corresponds  to  your  opinion  in  the  space  provided. 
Please  indicate  your  opinion  even  though  you  may  not  feel  strongly  about 
the  statement,  or  do  not  feel  well  informed.  Only  your  initial  perceptions 
are  sought.  Please  try  to  avoid  marking  a  "3"  (Cannot  agree  or  disagree) 
if  possible. 


Strongly 

1  Agree 

Tend  to 

Agree 

Can’t  Agree 
or  Disagree 

Tend  to 
Disagree 

Strongly 
Disagree  | 

1  1 

2 

3 

4 

5  1 

Using  the  scale  shown  above,  please  place  a  number  in  the  space 
provided  which  corresponds  to  your  opinion  about  each  statement 

_  29,  I  am,  in  general,  very  satisfied  with  the  manner  in  which  projects  and  tasks 

are  reviewed  and  ev^uated  in  our  division. 

_  30.  The  need  for  a  formal  project  selection  technique  is  not  great  relative  to 

other  changes  that  could  be  made  in  our  laboratory’s  project  review  and 
budget  allocation  process. 

_  31.  My  assessment  of  particular  projects  or  tasks  is  not  likely  to  change  as  a 

result  of  using  a  formal  project  selection  technique. 

_  32.  A  formal  project  selection  technique  would  help  to  make  my  budget 

recommendations  more  compatible  with  those  of  my  superiors. 

_  33.  A  formal  project  selection  technique  would  help  to  make  my  budget 

recommendations  more  compatible  with  those  of  my  research  staff. 

_  34.  A  formal  project  selection  technique  is  likely  to  increase  the  confidence  I 

have  in  my  budget  recommendations. 
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Strongly 

Agree 

Tend  to 

Agree 

Can't  Agree 
or  EKsagree 

Tend  to 
Disagree 

i 

Strongly 

Disagree 

1 

2 

3 

4 

5 

Using  the  scale  shown  above,  please  place  a  number  in  the  space 
provided  which  corresponds  to  your  opinion  about  each  statement. 


35.  A  formal  project  selection  technique  will  affect  the  type  of  information 
exchanged  when  I  review  and  evaluate  the  projects  and  tasks  in  my 
division/branch. 

36.  A  formal  project  selection  techniqtm  will  affect  the  amnnnt  of  information 
exchanged  when  I  review  and  evaluate  the  projects  and  tasks  in  my 
division/branch. 

37.  The  manner  in  which  we  review  and  evaluate  projects  and  tasks  in  our 
division  does  not  lend  itself  to  the  use  of  a  formal  project  selection  technique. 

38.  Formal  project  selection  techniques  are,  in  general,  difficult  to  use  and 
understand 

39.  The  information  generated  by  a  formal  prqject  selection  technique  cannot  be 
easily  communicated  to  others. 

40.  My  use  of  a  formal  project  selection  technique  is  likely  to  enhance  the  worth 
of  my  budget  recommendations. 

41.  My  iise  of  a  formal  project  selection  technique  is  likely  to  be  considered,  by 
my  immediate  superiors,  as  havii^  enhanced  the  worth  of  my  budget 
recommendations. 

42.  My  use  of  a  formal  project  selection  technique  is  likely  to  be  considered,  by 
my  research  staff,  as  having  enhanced  the  worth  of  my  budget 
recommendations. 

43.  More  relevant  information  is  likely  to  be  exchanged  in  my  division/branch  if 
I  use  a  formal  project  selection  technique. 
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1  strongly 

Tend  to 

Can’t  Agree 

Tend  to 

Strongly  1 

Agree 

or  Disagree 

Disagree 

Disagree  | 

1  ^ 

2 

3 

4 

_ 

Using  the  scale  shown  above,  please  place  a  number  in  the  space 
provided  which  corresponds  to  your  opinion  about  each  statement 


44.  A  formal  project  selection  technique  is  likely  to  help  identify  critical  or 
sensitive  aspects  of  my  projects  or  tasks. 

45.  My  use  of  a  formal  project  selection  technique  is  likely  to  increase  the  overall 
effectiveness  of  reviewing  and  evaluating  our  division’s  projects  and  tasks. 

46.  The  output  data  that  would  be  generated  by  a  formal  project  selection 
technique  could  not  adequately  meet  my  need  for  such  data. 

47.  I  have  some  reservation  about  the  way  in  which  formal  project  selection 
techniques  combine  or  manipulate  the  input  data. 

48.  1  would  find  it  extremely  difficult  to  obtain  the  input  data  necessary  to  use  a 
formal  project  selection  technique. 

49.  The  need  for  a  formal  project  selection  technique  is  very  much  increased 
when  funds  are  scarce. 

50.  I  would  recommend  that  a  formal  project  selection  technique  be  used  on  ^ 
of  the  projects  or  tasks  in  my  division. 

51.  I  would  recommend  that  a  formal  project  selection  technique  be  used  on  at 
least  50  percent  of  the  projects  or  tasks  in  my  division. 

52.  The  process  by  which  most  formal  project  selection  techniques  generate  their 
output  data  could  easily  be  explained  to  my  colleagues. 
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strongly 

Agree 


Tend  to 
Agree 

2 


Can’t  Agree 
or  Disagree 

3 


Tend  to 
Disagree 


Strongly 

Disagree 

5 


Using  the  scale  shown  above,  please  place  a  niunber  in  the  space 
provided  which  corresponds  to  your  opinion  about  each  statement. 

_  53.  I  personally  feel  that  R&D  prcgect  selection  decisions  which  are  based  on 

experience  and  expertise  will  yield  better  results  than  decisions  made  with 
the  use  of  a  formal  project  selection  technique. 

_  54.  The  output  data  generated  by  a  formal  project  selection  technique  could  be 

easily  interpreted  by  laboratoiy  decision  makers. 

_  55.  It  is  likely  that  a  formal  project  selection  technique  will  provide  common 

outputs  ^at  could  be  easily  understood  by  all  laboratory  decision  makers. 

_  56.  The  mathematics  involved  in  most  formal  project  selection  techniques  are 

more  complicated  than  is  necessary  for  our  project  selection  process. 

_  57.  The  input  data  for  most  project  selection  techniques  is  difficult  to  generate 

due  to  the  uncertainties  associated  with  technological  developments. 

_  58.  When  I  make  R&D  project  selection  decisions,  I  prefer  to  use  a  structured, 

qiiantifiable  process. 
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Part  HI 


59.  What  is  your  gender?  (Circle  one)  A)  Male  B)  Female 


60.  What  is  your  age? 

A)20>25 

B)  26-30 

C)  31-35 

(Circle  one) 

D)  36-40 

E)  41-45 

F)  46-50 

G)  51-55 

H)56  + 

61.  Please  print  your  military  rank  or  civilian  grade. 


62.  Which  Wright  Laboratory  organization  are  you  associated  with? 

A)  Command  Section 

B)  Operations  &  Support  Directorate 

C)  Avionics  Directorate 

D)  Sdid  State  Electronics  Directorate 

E)  Flight  Dynamics  Directorate 

F)  Assistant  Director,  Fmandal  Mgmt  and  (Comptroller 

G)  Materials  Directorate 

H)  Armament  Directorate 

I)  Manufacturing  Technology  Directorate 

J)  R&D  Contracting  Division 

K)  Aero  Propulsion  &  Power  Directorate 

L)  Plans  &  Programs  Directorate 

M)  Other 


63.  Which  organizational  level  best  describes  your  position? 
(circle  one) 

A)  Directorate 

B)  Division 

C)  Branch 

D)  Section 

E)  Other 


(Circle  one) 
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64.  What  is  the  highest  educational  level  you  have  obtained? 
(Circle  one) 

A)  High  School  Diploma 

B)  Associate’s  Depnee 

C)  Badielor’s  Degree 

D)  Master’s  Degree 

E)  Ph.D. 

F)  Other 


65.  What  were  your  major  fields  of  study  in  the  degrees  you’ve  obtained? 

A)  Associate’s  Degree  _ 

B)  Bachelor’s  Degree  ■ _ 

C)  Master's  Degree  _ 

D)  Ph.D.  _ 


66.  Have  you  ever  taken  a  course,  or  attended  a  seminar  in  either  management  science  or 
operations  research  techniques? 

(Circle  one) 

A)  Yes  B)  No 


67.  Would  you  be  willing  to  attend  a  course  or  seminar  on  formal  R&D  prqject  selection 
techniques?  (Circle  one) 

A)  Yes  B)  No 


68.  Have  you  ever  taken  a  coiuse  or  seminar  in  R&D  or  laboratory  management?  (Circle 
one) 


A)  Yes  B)  No 


69.  What  is  the  approximate  size  of  the  budget  you  are  responsible  for? 

$ _ 


70.  What  IS  your  estimate  of  the  percentage  of  your  division’s  budget  dedicated  to 
in-house  research  versus  contracted  research? 

In-house  % _  Contracted  % _  (Total  100%)  ■ 
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71.  What  is  your  estimate  of  the  percentage  of  your  budget  that  is  dedicated  to 
discretionary  research  efforts  which  are  chosen  by  your  division  for  their  potential  value 
to  future  Air  Force  weapons  systems,  as  opposed  to  research  efforts  which  are  specLfkally 
requested  by  an  Air  Force  system  program  office  (SPO)  or  another  outside  organization  to 
support  a  particular  weapons  system? 


Discretionary  research 

chosen  by  you  or  your 

division  % _ 

Research  specifically 
requested  by  a  SPO 

or  outside  organization  % _  (Total  1<X)%) 


THE  END 


THANK  YOU  FOR  TAKING  THE  TIME  TO  COMPLETE  THIS  SURVEY 
YOUR  COOPERATION  IS  GREATLY  APPRECIATED 
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jt^ndix  E;  Siirvev  Data  Base 


» 


The  survey  data  was  transferred  to  optical  scan  sheets  for  input  into  AFFTs 
computer.  The  following  printout  shows  the  survey  results  as  alphanumeric 
characters.  Please  refer  to  Appendix  D  for  survey  questions.  However,  in  an  effort 
to  maintain  the  survey  members  confidentiality,  the  column  format  and  it’s 
reference  survey  question  have  been  deleted.  This  information  is  available  for 
researchers  upon  request  pending  AFIT  approval. 
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00002052OAAAAOOOAOAODOOOOCBCBCBBDFONOBCCCBBBBCOBBCCBCDBCBOCBBBBCCBAFDEIBEBAACBE 

00002052 

000020S3ODODDDODDDODDDDDBCBCCCCCBGGNBBCEECCCOCDCCCCBCCCDDCCCBCCCCBAEOK 

00002053 

000020540DDDDDODDDDDDOOD  AFEU)U}BBBBAA 

00002054  1 

00002  OOODDDOOOODODDOOOOBDBBCBBCECHDADCCBBBCCBCEDBBBDBACEBCACCB&CABGCMDCBBADBG 
00002140 

00002139OAAAOOOOAAOOOADOAEOCEEFOF6FAAABOOEBOCOCEBEDOEDEACEEBABCBADAHHE  DO  ABEC 
00002139 

00002138DCADCCDDOADDDODOEBDDGGGDGAAACABODEBBOBDOCCDBDCBBOEEEAEECBOA6  I  DOB  ACAE 
00002138 

000021 3 7DODDDDDDDODDDODOAFODDODDDAANAACCCCCCECCCCCCCCCCCCCCCCCCCCCABMGIKEABACCH 
00002137 

00002136AOAAA  ADAAAODDDOGCCBFCGCOCCADACDOBCCBBBOBDBDBCEDBDDDCBBBABAELCFGDAAAEAA 
00002136 

00002135ODDAODDOOOODDDODBCBCCCCCOBBNBBDBBBBBODDBBBBBBDODBBDBOBBOOBAFLEKCDBAADAJ 

00002135 

00002134OOAAAADOODDODOOCBECCFFFOOOCAABBEDOBBOBDODBDEDCACBEDCADDCADAGMG  KDBAABGI 
00002134 

0000213 3DCODCAADOAOODOOODCBABBABDOBNBBBAAABBACOBBBBBBCBADDBDABBCBDAHI.GL  DAAADEA 
00002133 

00002132DAAACAAOACAODDDCCCCC  CCDOCCAOABCDBBBCBOCBBCBCDABCEDCBBDDBCAODR  6DABACAE 
00002132 

00002131AAAAAAAAAAAAAAAACGGGG  GGGDDABAA  EEEECCCECCCCCCCCCEECBCCCCEAEIGGIEABACE 
00002131 

00002130OOCDOOOOOOOOOOOOAC  BBBBBBBDNAAC0CAAADDDBB8AAA0BDBDBBEBBCBAAGHC  OEBAABBfl 
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Appendix  Fi  Interview  Guide/Response  Sheet 


Figures  55-57  are  reproductions  from  sections  of  a  large  blue  print  that  was 
shown  to  the  participants  during  the  5-7  minute  lateral  airfoil  overview.  Appendix 
C  details  the  depth  of  information  revealed  during  the  "brain  storming"  session 
that  followed.  Participant  determinant  attributes  were  recorded  on  the  margin  at 
the  bottom  of  the  first  page. 
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A.  You  are  given  an  engineering  development  cost  estimate  of  $210  million 
for  a  proposed  Lateral  Airfoil  research  project. 

1.  Mark  your  opinion  of  the  utility  of  the  cost  estimate  in  a  decision 
to  expedite,  shelve  or  abort  the  proposed  research  project. 


Low  Utility 


High  Utility 


2.  Which  of  the  following  positions  would  you  champion? 
(select  one) 

a.  Support  the  project 

b.  Table  the  project  until  some  later  date 

c.  Oppose  the  project 


3.  List  any  other  determinant  attributes  used  and  your  opinion  of 
their  utiUty. 


Low  Utility 


High  Utility 
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Low  Utility 


High  Utility 


3. 


Low  Utility 


High  Utility 


4. 


\ 


I.  I  .LI..  I _ I..  I _ I.  I  I _ I...  .1 _ _ I.  -/.--.I _ I _ f _ I  .  .  I.  ..!  ....L..,  _ i 


Low  Utility 


High  Utility 


Low  Utility 


High  Utility 


3. 


Low  Utility  High  Utility 


4. 


Low  Utility  Hi^  Utility 
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Interview  Survey  Form 


B .  Three  or  four  letter  office  S3anbol  _ 

0  ■  Please  mark  your  opinion  of  the  utility  of  the  following  decision  making 
attributes  using  the  line  mark  scales. 

1.  Time  Period:  the  estimated  project  completion  time 

.  f....  1..  ..l....  1... .  'I  ■■■I.. Ill  I.  . ..  I  ■  ■  ..I.. ..I..  ..I.  -  ..1.  .  ■■l.i.iln.il  '■■tl.inli  .111,  ..li  I  ..V-VH  lli.il...il  ■1..HH1.I 

Low  UtiHty  ffigh  UtUity 

2.  Resource  Availability:  availability  of  personnel,  equipment,  or  facilities 

Low  Utility  '  High  Utility 


3.  Tecbnical  Merit:  characterized  as  new  or  better  capabilities 


Low  Utility 


Hi^  Utility 
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4.  Ltikellhiood  of  Success:  probability  of  achieving  technical  success  within 
time  constraints 


Low  Utility 


ffighUtUity 


1 


5.  Ail*  Fotce  Need:  the  degree  the  Air  Force  hag  articulated  a  need 


litlllimhMtlll»i<lillHlH  llliilii.iiiiiilii.ili.il  (■..u'mi  Imti  mill  III  liliiliiiiliiiillllllii  iil/iiiliiiilnii  ll  inliiil 'iiiiliiiilinifcn  it 


Low  Utility 


BighUtiHty 


6.  Cost/Benefit  Ratio: 


Low  Utility 


ffigh  UtiUty 


* 
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MECHflNI 


Figure  55.  Mechanical  Linkages  for  the  Lateral  Airfoil  System 
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